---
title: "What an AI engineer actually does"
description: "Data, evaluation, operations and cost per request — where an AI engineer’s week actually goes, and what to learn first."
date: 2026-06-10T09:00:00.000Z
updated: 2026-08-06T10:33:58.915Z
url: https://eldan.ai/blog/what-ai-engineers-actually-do
locale: en
tags: ["AI & machine learning", "Career"]
---

In comments and in interviews the same question comes up: *how much mathematics do I need to become an AI engineer*. Fair question — but it describes the job about as accurately as "how much do I need to know about concrete" describes architecture.

Below is an honest breakdown of the working week, averaged across several product teams. The percentages are approximate; the order of magnitude is not.

## What the week is made of

1. Data: collection, cleaning, labelling, checking distributions — 30–40% of the time.
2. Evaluation: test sets, metrics, reading failures by hand — around 20%.
3. Service engineering: API, queues, caching, rate limits, cost per request — around 25%.
4. The model itself: architecture, fine-tuning, prompts — rarely more than 15%.
5. Talking to product: what counts as success and how we measure it — the remainder.

**Training the model is the most visible and the shortest part of the job.** Everything else decides whether the thing reaches production and survives its second month there.

## Why evaluation beats modelling

A model without an evaluation set is a hypothesis, not a product. Until you can answer "did that make it better or worse" repeatably, every prompt and hyperparameter change is a blind bet.

A minimal evaluation suite takes a day. Twenty real requests, the signals a correct answer must contain, and a `score` function — that is enough to stop arguing and start measuring.

```python title="eval/run.py"
from dataclasses import dataclass


@dataclass
class Case:
    prompt: str
    must_contain: list[str]


CASES = [
    Case("How much does delivery to Astana cost?", ["deliver", "Astana"]),
    Case("Return JSON with the fields name and price", ["name", "price"]),
]


def score(answer: str, case: Case) -> float:
    hits = sum(1 for token in case.must_contain if token.lower() in answer.lower())
    return hits / len(case.must_contain)


def run(model) -> float:
    return sum(score(model(case.prompt), case) for case in CASES) / len(CASES)
```

> **TIP:** Where to start
>
> Twenty labelled examples and one scoring script buy more than a week of hyperparameter search. It is the first thing worth doing on any AI project — and usually the last thing done.

## What this means if you are breaking in

- Python and SQL — to the point where unfamiliar code reads easily.
- A data pipeline: source → validation → storage → sampling. Be able to build the whole thing.
- One evaluation tool and one deployment path, practised until they are automatic.
- Mathematics as needed: linear algebra and statistics cover most of the work.

---

> An engineer who can measure quality is worth more than an engineer who can train models. The first one cannot be replaced by a library.

Practical tool walkthroughs go out on the [blog](/blog) and in the open materials on the [YouTube channel](https://www.youtube.com/@Eldan_nomad).