As AI translation becomes part of everyday localization workflows, the challenge is shifting from speed to trust. This article explains how Hansem QA combines AI quality estimation, rule-based QA, contextual AI review, and customer-specific validation to measure translation risk and manage high-volume multilingual content more reliably.
Better AI Translation Makes Verification More Important
Generative AI and large language models are rapidly improving translation quality. But in real-world localization projects, “the translation is generally good” is not enough.
For technical documentation, product manuals, websites, training materials, and other customer-facing content, a single mistranslation, omission, incorrect number or unit, or brand-inconsistent expression can become a serious quality issue.
At the same time, asking linguists to reread every AI-translated sentence from beginning to end limits much of the productivity gain AI is expected to provide. Reducing human review too aggressively creates the opposite risk: critical errors may reach the customer.
The better approach is not to review every translation at the same level. It is to measure quality risk first and adjust human review based on that risk.
This becomes especially important for global manufacturers managing large volumes of multilingual product documentation across multiple products, models, markets, and languages.
Translation Quality Evaluation Is Changing in the AI Era
Traditional translation quality evaluation has relied heavily on human reviewers reading translated content and identifying errors.
For high-volume localization projects, however, reviewing every segment can be impractical because of time and cost. Teams have therefore often relied on sampling to estimate overall quality.
AI-based Quality Estimation, or QE, changes this approach.
QE can analyze large volumes of translated content at the segment level, estimate where quality risks are most likely to occur, and identify which segments should receive closer human review.
Instead of applying the same level of review to every sentence, localization teams can focus their attention on higher-risk content.
Translation quality evaluation therefore becomes more than a final scoring exercise. It becomes a way to identify risk and prioritize human review.
AI QE is not primarily a replacement for human review. Its greater value is in enabling risk-based review.
QE and QA Serve Different Purposes
Quality Estimation and Quality Assurance are closely related, but they perform different functions.
Quality Estimation (QE) analyzes the source and translation without requiring a reference translation. It estimates quality and risk and helps determine which segments should receive priority review.
Quality Assurance (QA) identifies specific issues that need to be corrected, including:
- incorrect numbers or units
- missing or incorrect tags
- untranslated content
- terminology violations
- prohibited expressions
- formatting issues
- style guide violations
Put simply:
QE asks: “Where should we look first?”
QA asks: “What is wrong?”
QE is particularly useful for prioritizing review across high-volume translation. QA is designed to identify specific defects before delivery.
Hansem QA connects both functions within the same translation quality assurance workflow. QE first measures translation risk. QA then checks rules, terminology, context, and customer-specific requirements.
The objective is not simply to generate an AI quality score. The evaluation needs to lead to review, correction, and quality control.
When Is AI Quality Estimation Most Useful?
Immediately after a high-volume translation run: Teams can quickly assess overall quality and identify higher-risk segments before beginning full review.
Early in a translation project: Recurring error patterns can be identified before they spread across a larger volume of content.
When multiple translators or vendors are involved: QE can help identify quality variation and determine where additional review may be required.
After large-scale MT or LLM translation: Segments with a higher likelihood of mistranslation, omission, or meaning distortion can be prioritized for human post-editing.
Before final delivery: Reviewers can focus first on higher-risk content rather than rereading the entire document at the same level.
After customer corrections or quality claims: Teams can check whether similar errors may exist in other files or languages.
When introducing a new translation engine, prompt, or workflow: QE can help compare the new approach with the existing process and determine whether it delivers a measurable quality improvement.
A Single Score Is Not Enough to Manage Translation Quality
Asking an AI model whether a translation is good and receiving one overall score is not sufficient for production quality management.
Different models may evaluate the same sentence differently. More importantly, a single score does not tell a localization team:
- what type of error occurred,
- where the error occurred,
- how serious it is, or
- what needs to be corrected.
For quality measurement to become actionable, errors need to be classified by both type and severity.
Hansem QA uses an MQM-based error classification approach and applies standardized criteria and scoring formulas.
The result therefore provides more than an overall number. It can be used to determine correction priorities, identify recurring error patterns, and make review decisions across large volumes of multilingual documentation.
Production Experience Becomes a Quality Rule Set
Hansem Global’s translation quality assurance process did not begin with AI.
Through decades of technical documentation development and localization work, we have accumulated knowledge about recurring translation errors, common review issues, terminology problems, and product- and market-specific requirements.
That knowledge has been structured into assets such as:
- terminology databases
- style guides
- QA checklists
- customer-specific review criteria
Hansem QA extends these existing quality assets by connecting them to AI-assisted and rule-based verification.
The approach is therefore not simply to add another AI model to the localization workflow.
The more important step is to convert production knowledge into criteria that can actually be checked, and then apply those criteria consistently across high-volume content.
How Hansem QA Evaluates AI Translation
Hansem QA is an internal quality verification tool developed by Hansem Global and used in our actual translation and localization production environment.
It combines Quality Estimation and Quality Assurance and is applied to customer production work rather than being used only as a demonstration tool.
The QE process evaluates AI- or machine-translated content at the segment level and classifies identified issues into categories such as Accuracy, Fluency, Style, and Terminology. It also assigns severity levels of Critical, Major, or Minor based on impact.
Example | Critical Error
Source: Do not restart the device.
Translation: Reinicie el dispositivo.
Expected meaning: No reinicie el dispositivo.
Assessment: The negative “no” is omitted, reversing the meaning of the source.
AI Does Not Decide the Final Score Arbitrarily
An important distinction in Hansem QA is that the AI model does not simply assign an overall document score based on its own judgment.
The AI identifies errors and their severity. The final quality score is calculated according to a predefined formula.
Hansem QA currently applies the following penalty weights:
Critical: 25 points
Major: 5 points
Minor: 1 point
The result is normalized as error density per 1,000 source words so that documents of different lengths can be compared using the same framework.
The default passing score is 95, although thresholds can be adjusted according to project or customer requirements.
We Do Not Rely on a Single AI Judgment
One limitation of AI-based quality estimation is that different models may evaluate the same translation differently.
Hansem Global currently applies two AI models during QE and compares their evaluations of the same segments.
When both models reach similar conclusions, confidence in the evaluation can increase. When their judgments differ, the segment can be treated as a signal that additional review may be required.
The objective is not simply to increase the number of AI models.
The more useful approach is to treat model disagreement itself as a quality management signal.
In QA, We Separate What Rules Do Best from What AI Does Best
Once QE identifies quality risk, QA looks for specific errors.
Hansem QA does not use AI for every type of quality check.
Issues that software can evaluate deterministically—such as numbers, tags, punctuation, spaces, untranslated text, and Unicode problems—are handled through rule-based checks.
This approach is faster, more consistent, and avoids unnecessary AI processing.
Number verification, for example, does more than compare character strings. Hansem QA recognizes locale-specific decimal and thousands separators and compares the actual numeric value.
For example:
English: 1,000.50
German: 1.000,50
Although the notation is different, the system can recognize that the numeric value is the same.
Tags and variable placeholders can also be checked for missing elements, extra elements, incorrect sequence, and mismatched IDs.
AI is applied where semantic judgment is more useful, including:
- ambiguity
- context
- tone
- missing meaning
- added meaning
The principle is not to solve everything with AI. It is to use deterministic rules where the result can be determined reliably and use AI where contextual or semantic reasoning adds value.
Customer Terminology and Style Guides Become Part of QA
A translation can be grammatically correct and still be unsuitable for customer use.
Product names and UI terminology must be accurate. Brand voice must remain consistent. Rules for numbers, units, dates, punctuation, and preferred or prohibited expressions must also be followed.
Hansem QA checks terminology with rules-based logic first, then uses AI context review where simple string matching is not enough. For example, the word “save” can mean storing a file or reducing time or cost. AI context review helps distinguish these meanings and recognize synonyms or inflected forms in context.
Example | Terminology Error Missed by Simple Rule-Based QA
Approved term: login → iniciar sesión
Source: Please log in again.
Translation: Vuelva a acceder.
Rule-based QA: The termbase contains login, while the source uses log in. A simple string match may therefore miss the relationship.
AI contextual QA: The system recognizes that log in refers to the same concept as login and flags that the approved target term iniciar sesión was not used.
Style guides are also used as active evaluation criteria rather than passive reference documents. AI can read rules covering number and unit formats, UI wording, preferred and prohibited expressions, punctuation, and tone, then check whether the translation follows those requirements. This makes client-specific standards part of the quality evaluation itself.
Hansem QA Connects QE and QA in One Workflow
Within Hansem QA, QE and QA are not isolated functions. They form a connected quality verification workflow.
1. AI QE — Identify Risk and Review Priority
Without requiring a reference translation, AI QE analyzes source and target segments, estimates quality risk, and identifies content that should receive priority human review.
For large multilingual documentation programs, this allows reviewers to focus their time on higher-risk content instead of applying the same level of review to every sentence.
2. QA — Verify Actual Errors and Delivery Requirements
Deterministic issues such as numbers, dates, units, tags, markup, untranslated content, prohibited terms, length limits, formatting, and punctuation can be checked through rules.
Areas that require semantic interpretation—such as terminology, context, tone, and style guide compliance—can be supplemented by AI-assisted review.
QA therefore converts QE risk signals into specific issues that can be corrected while also checking critical delivery requirements across the full content set.
3. Customer-Specific Rules — Verify Requirements Beyond Standard QA
Some requirements apply only to a particular customer, product model, market version, section, or page.
Examples include:
- confirming that required regional regulatory text appears on a specific page
- checking that sections appear in the required order
- validating model names or part numbers against a defined format
Hansem can convert these project-specific requirements into automated validation logic and operate those checks for as long as the project requires.
These rules may have limited reuse outside the individual project, but they are often the requirements most closely tied to actual customer production.
The ability to convert project-specific requirements into practical validation logic is an important part of Hansem Global’s production engineering approach.
Back Translation Provides an Additional Diagnostic Layer
Some of the most difficult translation errors are sentences that read naturally but no longer preserve the original meaning.
Hansem QA can perform a separate back-translation review to help identify these cases.
The target translation is translated back into the source language, and AI compares the back translation with the original source to identify possible:
- omissions
- unnecessary additions
- meaning changes
- number and unit errors
- reversals of positive and negative meaning
Back translation is not treated as an absolute pass/fail criterion.
Because the back translation itself is also generated by AI, it can introduce additional errors.
Instead, Hansem QA uses back translation as an independent diagnostic method for identifying segments that may warrant further review.
Quality, Cost, and Processing Efficiency Must Be Managed Together
Applying AI in a real localization production environment requires more than accuracy. Cost and processing time also matter.
Hansem QA tracks the input and output tokens used during AI-based QA and QE, along with estimated cost. This allows teams to review both individual run costs and cumulative usage.
The data can help determine which AI models and evaluation methods are appropriate for a given project.
It is also why deterministic checks are processed locally and AI is called only where semantic evaluation is needed.
In production, using more AI is not automatically better.
The objective is to apply the right level of AI to the right quality problem.
The Next Challenge in AI Translation Is Verification
AI translation itself is becoming increasingly accessible.
The next point of differentiation may therefore be less about whether an organization can use AI for translation and more about whether it can measure, verify, and manage AI-generated output at a level that makes it reliable for real customer content.
This is particularly important for global manufacturers managing high-volume technical documentation, product manuals, service information, training content, and other multilingual product content.
Hansem Global structures production quality knowledge into terminology databases, style guides, QA rules, and customer-specific validation requirements. We connect these assets with AI quality estimation, rule-based QA, contextual AI review, and back-translation analysis.
The goal is not to maximize AI usage.
It is to make AI-generated translation reliable enough for production.
If quality can be measured, it can be managed. And if it can be managed, AI translation can be applied at scale with greater control and confidence.
That is the direction behind the quality technology we continue to develop at Hansem Global.