Revised & Improved Version:

When people hear "AI," they often think of ChatGPT or other generative AI models categorized as LLMs (Large Language Models). However, a new AI paradigm is gaining attention—Agent AI, which I’ll explore in this article.

The Four Categories of AI

Before diving into Agent AI, let's first classify AI based on how it is trained and its purpose:

1. Supervised AI – This type of AI learns from labeled data, meaning humans (or other intelligent entities) provide direct input-output mappings. Common examples include email spam classification and image recognition. Interestingly, the first version of ChatGPT was a supervised AI model, requiring human-written questions and answers for training.
2. Unsupervised AI – Unlike supervised AI, this category doesn't require manually labeled data. It learns from patterns in raw data. Examples include image recognition for identifying individuals and the latest versions of ChatGPT, which were trained on large-scale Q&A data extracted from web forums.
3. Semi-Supervised AI – This hybrid approach uses both labeled and unlabeled data. A good example is recommendation systems that suggest videos or products based on user interactions.
4. Reinforcement Learning (RL) – This is where Agent AI comes into play. RL differs from other AI types because it focuses on learning complex actions while optimizing for an objective.

Reinforcement Learning & AI Agents

The foundation of RL is built on multiple theories, enabling its real-world applications in autonomous vehicles, aircraft autopilots, and trading agents. The key concepts behind RL include:

1. Markov Decision Process (MDP) – A framework for making optimal decisions in uncertain environments by analyzing sequential data.
2. Multi-Armed Bandit Problem – A decision-making problem where an agent must choose the best action from a fixed set. In trading, for example, an agent may decide between holding, buying, or selling at each time step.
3. Artificial Neural Networks (ANNs) – A fundamental component of modern AI models, simulating the human brain’s decision-making process using multiple linear equations. ANNs allow AI to handle complex decision-making tasks.

Training an AI Trading Agent

For an RL-based trading agent, rewards act as fuel, guiding the AI toward its objective. The most basic reward function is ROI (Return on Investment) since maximizing profit is the primary goal. However, using ROI alone can lead to overfitting, where the agent performs well in training but fails in real-world trading. To prevent overfitting, additional penalties and rewards must be incorporated into the reward function, such as:

• Overtrading penalty – To discourage excessive trading.
• Drawdown penalty – To reduce exposure to high-risk scenarios.
• Sharpe ratio reward/penalty – To encourage stable, risk-adjusted returns.

Can Trading Agents Generate Buy/Sell Signals?

Many traders expect AI trading agents to generate explicit buy/sell signals. However, most RL-based trading agents follow a multi-armed bandit approach, meaning they can only take predefined actions like hold, buy, or sell.

To generate buy/sell signals with target prices, we could introduce additional actions, such as:

• Buy signal at +2.5% price increase
• Buy signal at +5% price increase
• Buy signal at +10% price increase
• Sell signal at -2.5% price decrease
• Sell signal at -5% price decrease
• Sell signal at -10% price decrease
• Hold

However, increasing the number of actions requires significantly more data and training steps, making the model more complex. In practice, using an agent for direct auto-trading is often a better approach than relying on discrete trading signals.

Key Inputs for AI Trading Agents

For training a trading agent, I recommend starting with technical indicators and time-based features. As you advance, you can incorporate feature engineering, news sentiment analysis, and correlated assets to improve performance.

Conclusion

AI trading agents leverage reinforcement learning to make buy, sell, or hold decisions based on trained data. While they excel at fully automated trading, they aren't typically designed for sending explicit trade signals unless specifically programmed to do so. With the right input features, reward functions, and risk management strategies, AI agents can offer a powerful, automated approach to crypto trading.