[INCOMPLETE] Thoughts on AI

[Someday], 2026

One of the main issues with modern AI discourse, specifically with those unfamiliar with the mechanisms of AI, is the mystical thinking surrounding its operation and implementation. In this article, I would like to offer my personal mental model of said mechanisms, which I hope will be useful towards the pursuit of understanding AI. In particular, my understanding of AI systems is rooted in the language of interpolation. Using this description, I hope that we can more precisely discuss the powers, potential applications and limitations of AI systems.

The fundemental problem formulation

I consider the fundemental problem of AI to be formulated as follows: For any underlying task that an AI can be designed to work on, it can be modelled functionally. Specifically, we can analyze any task as some numerical input output system. For example

Image Classification: input image (matrix of pixel intensities), output class
Text Based Generative AI: input prompt text (tokenized to numbers), output response text, or perhaps input previous text in window, output next text in window (depending on the architecture)
Reinforcement Learning: input state (vector of features), output action
...etc...

The fundemental problem, hence, is to find the underlying function which describe the optimal mapping from inputs to outputs. In particular, the information by which gives the machine learning algorithm correctness is data, rather than some logical insight from the programmer (as in classical computing). I think this formulation is particularly useful because it provides insight into a few key aspects of AI systems, for example

What does training mean, and why does it need so much data?
What are model weights and why are they critical to the operation of the model? Why are weights not transferrable?
Intuiatively, why is AI such a computationally intensive process, both in training and inference?
What is the upper bound of performance from machine learning based algorithms?

These themes will be central to the exploration in the following sections.

Solving the Fundemental Problem

Generally, to design an AI system/architecture, the designer (possibly impliclty) simply makes a guess at the parameterized function governing the process.

For example, I can look at the below data, (ignore the line, just look at the dots)

A good guess would be that the underlying function (represented by the line in the above image) to fit this data may be $f_{\vec w}(x)=w_4x^4+w_3x^3+w_2x^2+w_1x+w_0$ where $\vec w=\{w_i\}_{i\in[0,4]}$ are unknown parameters. In AI terminology, we must "train" the AI to find the "parameters", or "weights", of the model.

To train the AI model, we must solve an optimization problem to find the best parameters. In partiuclar, we solve

\min_{\vec w}e(f_{\vec w}(\vec x), \vec y)

where

\vec x, \vec y

is the labelled data inputs/outputs respectively, and

\vec w

are the parameters of the function

f

, and

e

is some error metric (for example, mean squared error). The error metric is very important as it dictates not only what is considered "the function matching the data's outputs", but also how easy the optimization problem is to solve (outside of scope of this article).

An obvious observation is that not all parameterized functions will work. In the example above, a linear function would not be able to fit the data, and hence would not be a good guess. We can mathematically describe what it means for a parameterized function to be a "good guess" for a certain set of data: if the function can arbitrarily fit the data, then it is a good guess, in particular

\forall\epsilon,\exists\vec w, \text{such that } e(f_{\vec w}(\vec x), \vec y) < \epsilon

Since error functions are generally defined to be non-negative, this guarentees that

f_{\vec w}

can be made to fit the data arbitrarily well.

We can extend on this idea by considering if a parameterized function can fit any data, namely

\forall\vec x, \vec y, \forall\epsilon,\exists\vec w, \text{such that } e(f_{\vec w}(\vec x), \vec y) < \epsilon

By conducting this extension, we can see that the parameterized function is not only a good guess for a specific set of data, but also a good guess for any data. This process is important because it is not necessarily possible to be able to visualize and guess a more efficient (with respect to number of model weights, hence complexity of function) function.

AI in the context of existing computational models and algorithms

One of the most, I believe, important questions I have had to ask regarding AI systems is the following: What differentiates a "AI" computation from other types of computations? In particular, as someone who has spent some time developing "classical" algorithms, I believe that these algorithms are also artificial systems which demonstrate intelligence (AI?). Something to note is that it is possible to develop AI systems which solve these "classical problems". However, instead of having the specifying the developer specify the computational operations comprising the algorithm, the AI system learns the function through examples, and attempts to fit to it.

This, however, is clearly an absurd construction. It is obvious that nobody is seeking to develop AI to solve problems solvable with classical algorithms in the vast majority of cases. This is because of the computational resources required to run AI systems. Just as classical algorithms can act as a way of more compactly representing the same computation conducted by AI systems, AIs can be seen as a compact representation of the KNN inference machine.

Ultimatetly, I argue that machine learning can offer perfect performance, given infinite compute and data and perfect perception. In particular, this implementation would not even need to be particularly clever: by looking up your state obtained from your perfect perception in your infinite dataset, an example can be found which will offer the correct solution. This mirrors a principle in classical computation, where if you have a verifier for a given problem (namely given an input and solution it tells you if the solution is correct or not), it is possible to brute force all permutations of outputs and verify. Hence, I consider the nontriviality of developing machine learning algorithms to be in the same vein as developing classical algorithms: how do you find the correct answer with not only limitations in computational resources, but also in data and perception.

The Fundemental Bottleneck: Computation Resources

The theme of compactness or compression as being equivalent to intelligence is not a new idea.

Extrapolation and Human Level Intelligence

I believe that in order for an AI system to achieve human level intelligence, it must be able to extrapolate, which I claim is the same process as what people commonly describe as "creativity".

How does human intelligence achieve extrapolation?

random bullshit in brain theory

Trivializing AI research

The reason I wrote this article is because I believe that . However, in the pursuit of simplifying AI to be understandable, I hope not to leave an impression of trivialization upon this field of research. Firstly, even if superhuman, or even human level performance is not achievable by AI systems, it is still extremely non-trivial to develop computation systems made possible through AI. There are still significant uses for systems whose behaviours are not represented through computer operations but rather through the learned patterns and representations (in interpolation tasks). I am personally of the belief that superhuman AI performance is possible: in particular if hardware is developed unconstrained by the limitations of human biology, it is entirely possible to support the sufficient computation required for super human level intelligence.