What an AI engineer actually does

In comments and in interviews the same question comes up: how much mathematics do I need to become an AI engineer. Fair question — but it describes the job about as accurately as "how much do I need to know about concrete" describes architecture.

Below is an honest breakdown of the working week, averaged across several product teams. The percentages are approximate; the order of magnitude is not.

What the week is made of

1. Data: collection, cleaning, labelling, checking distributions — 30–40% of the time.
2. Evaluation: test sets, metrics, reading failures by hand — around 20%.
3. Service engineering: API, queues, caching, rate limits, cost per request — around 25%.
4. The model itself: architecture, fine-tuning, prompts — rarely more than 15%.
5. Talking to product: what counts as success and how we measure it — the remainder.

Training the model is the most visible and the shortest part of the job. Everything else decides whether the thing reaches production and survives its second month there.

Why evaluation beats modelling

A model without an evaluation set is a hypothesis, not a product. Until you can answer "did that make it better or worse" repeatably, every prompt and hyperparameter change is a blind bet.

A minimal evaluation suite takes a day. Twenty real requests, the signals a correct answer must contain, and a score function — that is enough to stop arguing and start measuring.

from dataclasses import dataclass


@dataclass
class Case:
    prompt: str
    must_contain: list[str]


CASES = [
    Case("How much does delivery to Astana cost?", ["deliver", "Astana"]),
    Case("Return JSON with the fields name and price", ["name", "price"]),
]


def score(answer: str, case: Case) -> float:
    hits = sum(1 for token in case.must_contain if token.lower() in answer.lower())
    return hits / len(case.must_contain)


def run(model) -> float:
    return sum(score(model(case.prompt), case) for case in CASES) / len(CASES)

What this means if you are breaking in

• Python and SQL — to the point where unfamiliar code reads easily.
• A data pipeline: source → validation → storage → sampling. Be able to build the whole thing.
• One evaluation tool and one deployment path, practised until they are automatic.
• Mathematics as needed: linear algebra and statistics cover most of the work.

An engineer who can measure quality is worth more than an engineer who can train models. The first one cannot be replaced by a library.

Practical tool walkthroughs go out on the blog and in the open materials on the YouTube channel.