Week 4: What AI Can and Cannot Do

 

Welcome to Week 4 of the AI Literacy Course.

In earlier lessons, we explored where AI appears in everyday life, how different AI systems work, and why language models can produce fluent answers that are still wrong.

Now we turn to a practical question:

When is AI useful, when does its output need careful checking, and when must people remain in control?

The answer depends on the task, the available data, the people affected, and what could happen if the system is wrong.

Learning goals

By the end of this lesson, learners should be able to:

• Describe an AI capability as a specific task
• Distinguish capability, usefulness, and reliability
• Explain why AI strengths also create limitations
• Classify an AI use as green, amber, or red
• Recognise automation bias
• Choose the smallest safe role for AI

1. Capability depends on the task

AI can perform well at one task and poorly at another, even within the same application.

A chatbot may rewrite a paragraph clearly but calculate a complicated tax case incorrectly. It may create useful study questions but invent a source. An AI system may recognise common objects in photographs but perform less reliably with unusual images or poor lighting.

Instead of asking only, “Is this AI good?”, ask more specific questions:

• What exact task is it performing?
• What information does it have?
• Who will use the output?
• Who could be affected?
• What could happen if it is wrong?

A single impressive answer does not prove that a system is reliable for every task.

2. Capability, usefulness, and reliability

These three ideas are connected, but they are not the same.

Capability means that an AI system can produce a particular kind of output.

Usefulness means that the output helps someone complete a task.

Reliability means that the system performs consistently well enough for that task and context.

For example, a chatbot may be capable of explaining a medical term. Its explanation may help someone understand difficult vocabulary. That makes it useful.

However, this does not mean that the chatbot is reliable enough to diagnose an illness or recommend treatment.

An AI output can therefore be useful without being reliable enough for an important decision.

3. What AI commonly does well

Different AI systems have different abilities. Depending on the system, AI can often help people:

• organise information;
• rewrite or simplify text;
• translate between languages;
• generate ideas and first drafts;
• create practice questions;
• summarise documents;
• classify information;
• identify statistical patterns;
• estimate probabilities.

These capabilities can save time and make information more accessible.

For example, a learner might ask AI to explain a difficult paragraph in simpler language, suggest headings for an assignment, or create questions for revision.

But the learner still needs to compare the output with the original material. A simplified explanation may leave out an important detail, and a generated practice question may contain an error.

4. What users cannot safely assume

AI output can appear polished, confident, and complete. That appearance can be misleading.

Users cannot safely assume that an AI system:

• has checked every claim against reliable evidence;
• understands the full local situation;
• has included every important perspective;
• performs equally well for every language or group;
• knows which human values should guide a decision;
• understands emotions as another person does;
• will protect sensitive information in every tool;
• can accept responsibility for the result.

This does not mean that AI is useless. It means that its output must be judged according to its purpose and possible consequences.

Confidence, detail, and professional language are not proof that an answer is correct.

5. Strengths and limitations travel together

Many AI strengths have related limitations.

Speed can help people complete routine work quickly. But it can also spread mistakes quickly.

Summarisation can make a long document easier to read. But a summary may remove uncertainty, disagreement, or minority perspectives.

Language support can make difficult material more accessible. But simplification or translation may change technical meaning.

Prediction can estimate what is likely based on past data. But future conditions may differ from the past.

Pattern recognition can identify similarities across large amounts of data. But a pattern does not automatically explain why something happened.

The important habit is to consider the benefit and the limitation together.

6. Data and context matter

AI systems depend on the information used to develop them and the input available during use.

Their performance may be affected by:

• incomplete or outdated data;
• incorrect examples;
• groups or languages that are poorly represented;
• unclear instructions;
• missing local information;
• conditions that differ from the original testing environment.

A system that works well on average may still perform poorly for a particular person, language, workplace, or community.

Local knowledge is also important. An AI-generated study plan may look excellent but ignore work schedules, family responsibilities, disability, available resources, or course requirements.

Always ask:

• Does the system have the information needed for this task?
• Does it work well in this language and context?
• Who was included when the system was tested?
• Who may be harmed if the output is wrong?

7. The traffic-light model

The traffic-light model helps us decide how much human checking and control an AI use requires.

The colour applies to a specific use in a specific situation. It does not describe the entire AI tool permanently.

Green: bounded support

Green uses are generally low-risk. Mistakes are usually easy to notice and correct.

Examples include:

• brainstorming possible essay topics;
• creating a personal checklist;
• suggesting alternative titles;
• generating practice questions;
• rephrasing your own paragraph for clarity.

AI supports the person, but the person still reviews the result.

Amber: verify carefully

Amber uses involve facts, advice, interpretation, assessment, or information that matters.

Examples include:

• summarising an article for an assignment;
• explaining a medical, legal, or financial topic;
• translating important workplace information;
• giving feedback on schoolwork;
• interpreting rules or regulations.

In amber situations, the user should check reliable sources, compare the output with the original material, and look for missing context.

Red: accountable human decision

Red uses involve decisions that may seriously affect a person’s rights, health, safety, education, employment, or access to services.

Examples include decisions about:

• medical treatment;
• employment or dismissal;
• school grades;
• eligibility for public support;
• legal rights;
• personal safety.

AI may assist an authorised professional, but it should not silently replace accountable human judgement. People affected by a decision should be able to understand it, question it, and request correction.

8. The same task can change colour

An AI activity is not always green, amber, or red in every situation.

For example:

• Summarising your own study notes may be green.
• Summarising a scientific article for an assessed assignment may be amber.
• Summarising medical records to guide treatment requires professional review and strong safeguards.

The task may appear similar, but the purpose, information, people affected, and consequences are different.

The greater the possible harm, the stronger the human review must be.

9. Automation bias

Automation bias means accepting an automated result too readily because it appears neutral, systematic, or authoritative.

For example, a teacher may give too much trust to an AI-generated assessment because the system presents a detailed score. A worker may follow an automated recommendation even when local experience suggests that something is wrong.

Automation bias can happen even when people know that AI makes mistakes. Time pressure, professional-looking output, and trust in technology can make people less likely to question the result.

To reduce automation bias:

• examine the evidence;
• compare the output with other information;
• look for missing context;
• ask what assumptions the system made;
• keep responsibility with an accountable person.

Human review must be meaningful. It should not consist of simply approving whatever the system produces.

10. Privacy is part of the decision

AI tools may process prompts, documents, images, account information, and other data.

Before entering personal or sensitive information, ask:

• Is this tool approved for the task?
• Is the information necessary?
• Could I remove names or identifying details?
• Could the task be completed with less data?
• Do I have permission to use this information?

Convenience is not permission. Personal data, confidentiality, GDPR, and the A–P–K method will be examined more closely in Week 15.

11. Choose the smallest safe AI role

Responsible AI use does not mean using AI for everything. It also does not mean rejecting AI completely.

A better approach is to choose the smallest safe AI role.

For example:

• AI may suggest ideas, but the learner chooses which idea to develop.
• AI may draft a summary, but the learner checks it against the original source.
• AI may identify a case for review, but a qualified person makes the decision.
• AI may explain a difficult concept, but it should not replace learning and understanding.
• AI may help prepare feedback, but the teacher remains responsible for assessment.

This approach allows people to benefit from AI while keeping evidence, judgement, and responsibility where they belong.

Classroom activity: Traffic-light analysis

Choose three or four examples of AI use from education, work, or everyday life.

Complete the following analysis for each example:

Exact AI task	Possible benefit	What could go wrong?	Green, amber, or red?	Necessary safeguard	Smallest safe AI role
Example 1
Example 2
Example 3
Example 4

Be prepared to explain why you chose each colour. Remember that another group may classify the same use differently if they imagine a different context.

Reflection questions

1. Why should AI capability be connected to a specific task?
2. What is the difference between capability, usefulness, and reliability?
3. Why can speed be both a strength and a limitation?
4. What important information can a summary remove?
5. Why can the same AI task change from green to amber or red?
6. What is automation bias?
7. Why is human review especially important when the consequences are serious?
8. What is the smallest safe AI role in one situation from your own life?

Key vocabulary

Capability:
What an AI system can do for a particular task.

Usefulness:
How an AI output helps someone complete a task or reach a goal.

Reliability:
How consistently an AI system performs well enough for a particular purpose and context.

Limitation:
A condition in which an AI output may be incomplete, incorrect, unfair, unsafe, or unsuitable.

Verification:
Checking an output against reliable evidence, original sources, local context, and human knowledge.

Consequence:
What may happen to people or situations if an output is used or a decision is wrong.

Automation bias:
Accepting an automated result too readily because it appears systematic or authoritative.

Traffic-light model:
A practical method for classifying an AI use as green support, amber verification, or a red human decision.

Smallest safe AI role:
Giving AI only the limited role that is suitable for the task while keeping review and responsibility with people.

Summary

In Week 4, we learned that AI capability is task-specific. An AI system may be capable and useful without being reliable enough for an important decision.

AI can support organisation, language, creativity, classification, and prediction. But users cannot assume that every output is true, complete, fair, suitable, or safe.

The traffic-light model helps us match human review to the possible consequences:

• Green for bounded, low-risk support
• Amber when verification is necessary
• Red when accountable human judgement must remain in control

Lesson 4 Interactive Quiz: What AI Can and Cannot Do

Choose one answer for each question. Then select Check my answers. You will receive feedback for every question. This practice quiz does not collect names or scores.

1. What is the difference between AI capability and reliability?

A. Capability concerns speed, while reliability concerns cost. B. Capability applies to machines, while reliability applies only to people. C. Capability is what a system can produce; reliability is whether it performs consistently well enough. D. Capability and reliability mean exactly the same thing.

2. A learner asks AI to simplify a difficult scientific paragraph. What is the best next step?

A. Compare the explanation with the original paragraph. B. Assume the simpler version contains every important detail. C. Submit the explanation as personal work without reading it. D. Reject the explanation because AI cannot support learning.

3. Why do strengths and limitations “travel together”?

A. Every useful AI output contains exactly one factual error. B. AI strengths disappear when people check the output. C. A limitation matters only when an AI system works slowly. D. A useful capability can create a related risk, such as a summary losing important detail.

4. Which example best fits the green category?

A. Using AI to decide whether someone should receive medical treatment. B. Using AI to create practice questions from your own study notes. C. Using AI as the only source for interpreting an employment contract. D. Allowing AI to assign final grades without teacher review.

5. A learner uses AI to summarise an article for an assessed assignment. Which category is most suitable?

A. Green, because every summary is low-risk. B. Red, because AI must never summarise text. C. Amber, because the summary must be checked against the original article. D. No category, because schoolwork is outside the traffic-light model.

6. An AI system recommends rejecting a job applicant. What is the smallest safe role for AI?

A. AI may flag information for review, but an accountable person makes the decision. B. AI makes the final decision if its score appears precise. C. AI rejects the applicant unless the applicant proves the system wrong. D. AI makes the decision, while a person signs the result without reviewing it.

7. Which situation is the clearest example of automation bias?

A. A learner compares an AI explanation with the textbook. B. A teacher questions an AI score after noticing missing context. C. A worker asks a colleague to review an unusual recommendation. D. A manager accepts an AI recommendation mainly because it looks objective and detailed.

8. An AI tool works well in testing but performs poorly for one local language group. What is the best explanation?

A. People in that group must be using the tool incorrectly. B. The training or testing data may not represent that language and context adequately. C. Average accuracy guarantees equal performance for every group. D. Local context affects people but cannot affect AI performance.

9. Before uploading a document containing personal information to an AI tool, what should you do?

A. Upload it if the tool produces a privacy-looking symbol. B. Upload the complete document so the AI has as much data as possible. C. Check approval and permission, and use less or de-identified data where possible. D. Upload it if you plan to delete the conversation later.

10. Why can the same AI activity be green in one situation and amber or red in another?

A. Its purpose, data, affected people, and possible consequences may change. B. The traffic-light colour depends mainly on the user’s favourite AI tool. C. An activity becomes safer whenever the AI answer is longer. D. Every AI activity should eventually be classified as green.

Reflection: Choose one AI use from your studies, work, or everyday life. What is the smallest safe role AI should have in that situation?

The right question is not only:

“Can AI do this?”

The better question is:

“What role should AI play here, and what safeguards are needed?”