# What Is AutoML? What Automated Machine Learning Does — and What It Doesn't Check Canonical: https://yourcloudblog.com/blog/what-is-automl/ Published: 2026-10-15 Author: YourCloudGroup · Publisher: YourCloudGroup · Category: Predictive AI Tags: automl, automated machine learning, model selection, model evaluation, target leakage > AutoML (automated machine learning) tries many model types and settings on your data and picks the best. It does not decide whether your columns are honest or your test is fair. What to check, with measured examples. **Short answer:** AutoML automates the repetitive part of building a predictive model — preparing columns, trying many algorithms and settings, and ranking them on a test set. It cannot know whether a column was recorded after the outcome, whether the test set was untouched, or which mistakes cost your business most. In a measured example, one leaked column lifted a model's score from 0.81 to 0.96 — and an automated search would happily pick it. **AutoML (automated machine learning)** is software that builds predictive models with little or no manual tuning. You give it a table of past cases with a known outcome; it prepares the columns, trains many different model types with many different settings, scores each one on data held back from training, and hands you the best performer. What used to take a data scientist days of trial and error takes minutes. What AutoML automates is the **search for a good model**. What it cannot automate is the **judgement about whether the data and the test are honest** — and that judgement decides whether the winning score means anything. ## What AutoML automates - **Column preparation:** turning categories into numbers, filling or flagging missing values, scaling. - **Algorithm choice:** trying several model families — logistic regression, random forests, gradient boosting, neural networks — on the same data. - **Settings (hyperparameters):** trying many configurations of each model, such as tree depth or learning rate. - **Ranking:** scoring every candidate on held-back data and choosing a winner, often presented as a leaderboard. Tools that offer AutoML include Amazon SageMaker Canvas, Google Vertex AI, Azure Machine Learning, DataRobot, H2O Driverless AI, KNIME's AutoML components and [YourCloudGroup Model Manager](https://yourcloudblog.com/model-manager/) — compared side by side in [which tools build predictive models from a CSV file](https://yourcloudblog.com/blog/tools-that-build-predictive-models-from-csv-files/). **How fast is it?** In one measured run, Model Manager's tournament of twelve model variants took about **4–5 minutes** on 60,698 training rows. ## What AutoML does not decide for you An automated search optimises whatever score it is given, on whatever data it is given. Five questions remain yours — or your tool's, if it checks them: ### 1. Will every column exist when you need the prediction? A column recorded after the outcome — a cancellation reason, the length of a sales call — makes every model look better, so an automated search will happily favour it. In a measured example on a public bank-marketing dataset, keeping one such column (call duration) lifted the winning model's score (ROC AUC) from **0.812 to 0.955**; none of that gain exists in real use. See [what is target leakage](https://yourcloudblog.com/blog/what-is-target-leakage/). ### 2. Was the test set truly untouched? If the same held-back rows are used again and again to compare and tune models, the winner is partly chosen *for* that test set, and its score flatters it. A fair process seals the test set before any model is trained and records every time it is reused. See [what is held-out model evaluation](https://yourcloudblog.com/blog/what-is-held-out-model-evaluation/). ### 3. Is the score the right one for your decision? Leaderboards often default to accuracy. On data where one outcome is rare, accuracy rewards models that ignore the rare outcome: in a measured example, the rule with the best accuracy (90.3%) found 244 of 928 subscribers, while a rule with lower accuracy found 579. See [balanced accuracy vs. accuracy](https://yourcloudblog.com/blog/balanced-accuracy-vs-accuracy/). ### 4. Can the result be repeated? If re-running the same search gives a different winner or a different score, the number on the leaderboard is partly luck. Repeatability has to be checked, not assumed. ### 5. What happens after the data changes? AutoML picks the best model for the past it was shown. When customers or markets shift — in the bank data, conversion rose from 11% to 31% between periods — the model needs to be re-checked against new outcomes. AutoML tools differ in how many of these they check. Some screen for leakage (DataRobot, for example, documents automatic leakage detection); others leave it entirely to the user. When choosing a tool, ask which of these five it checks and how it reports what it could not check. ## How YourCloudGroup Model Manager approaches AutoML [YourCloudGroup Model Manager](https://yourcloudblog.com/model-manager/) runs an automated **tournament of twelve model variants across six model types** (neural networks in three shapes, logistic regression, random forest and gradient boosting) and chooses the winner on **balanced accuracy and macro F1**, not raw accuracy. Around the tournament it handles the five questions above: - it **screens every column for leakage** on the training rows only — single columns, groups of columns, identifiers and post-event fields — before any model is trained; - it **seals the test set** with a hashed record and discloses any reuse; - it **verifies repeatability** — in one test, four full-data rebuilds of a 60,000-row neural network gave byte-identical predictions; - it evaluates models **against outcomes supplied later**; - it issues a **[certificate](https://yourcloudblog.com/blog/what-it-means-to-certify-a-machine-learning-model/)** of independent claims, marking anything it could not establish as "not assessed" rather than assuming it. ## Takeaway AutoML removes the slow part of model building — the search — and that is genuinely useful. It does not remove the need to check that the data is honest and the test is fair. Judge an AutoML tool not by its leaderboard but by what it checks before the leaderboard is drawn.