Who Decides What Is True?

From Fact-Checkers to Algorithms: Why a Label Is Not the Same as Truth

A small label appears on the screen: “false,” “misleading,” “missing context.” Perhaps it was added by a professional fact-checker, perhaps by a platform, perhaps by a community of users. For most people, the moment is almost imperceptible. But something important has happened. The claim has not merely received an answer. It has received a status.

That distinction is crucial. Truth itself has no button, badge, or colored label. A label is always assigned by someone: a journalist, a newsroom, an expert institution, a fact-checking organization, a platform, or a crowd of users. Sometimes the label is well justified. Sometimes it is partly correct. Sometimes it misses the problem. Sometimes it later has to be corrected. And that is precisely why it is dangerous when we begin treating a fact-checker as the final authority on truth.

We should not trust a fact-checker because they are called a fact-checker. We should trust the evidence they show, the methodology we can inspect, and their willingness to correct their own mistakes. In this sense, a fact-checker is simply another source. They can be a very good source. But they can also be biased, selective, or wrong. Their label is not proof in itself.

Truth, evidence, and an institutional label are not the same thing

First we need to separate things that contemporary information culture often compresses into one category. Something can be true but not yet proven. Something can match the prevailing scientific consensus while that consensus later changes. Something can be politically contested without being factually false. Something can be inaccurate but still lawful. And something can be correctly labeled false while the way it was selected for checking is politically or editorially biased.

This is not a semantic detail. If we confuse truth, provability, consensus, legality, and institutional labeling, we create a system in which “fact-checked” begins to mean almost the same as “true,” “unverified” almost the same as “suspicious,” and “contrary to consensus” almost the same as “false.” In such a system, epistemology quietly turns into administration.

A good fact-check should do the opposite. It should show what we know, what we do not know, which source supports which claim, and where interpretation begins. International IFCN and EFCSN standards therefore require transparency of sources, funding, methodology, and corrections, while emphasizing the use of primary sources. That makes sense. But a code of standards is not proof that every individual judgment is unbiased.

Bias can begin with the question: what will we check at all?

When we talk about bias among fact-checkers, we often look only at the final step: did they rate two similar claims differently? But the most important bias may arise earlier. Who decides which statement reaches the fact-checking desk in the first place?

That is selection power.

A study published in the Journal of the European Economic Association found different political “slants” in the content selected by French fact-checkers. Newsrooms checked different political sides with different intensity, and their selection partly followed the political orientation of the media outlet. At the same time, among 232 statements checked by more than one fact-checker, the researcher found only two cases of disagreement. That is an important result: the problem is not necessarily that fact-checkers systematically apply different standards to the same claim. A large part of the bias may appear in which claims they choose to check.

For the public, this is almost invisible. If one political side is checked five times more often than another, each individual check can be methodologically correct and yet the overall flow can create an asymmetric picture of the world. Readers do not see the mass of statements that were never selected. They see only the archive of fact-checks and infer from it who lied more often.

But here too, we should not generalize too quickly. A study of U.S. members of Congress for the 2018–2021 period did not find that Republican politicians were systematically checked more frequently than Democrats. It did, however, find another pronounced asymmetry: 20 percent of politicians received roughly 90 percent of all checks. Fact-checking is therefore heavily concentrated on the most prominent actors. This too means the archive of checks is not a neutral photograph of the entire political reality.

The correct conclusion is therefore not: “all fact-checkers are left-wing” or “all fact-checkers are right-wing.” The correct conclusion is more demanding: fact-checking is a journalistic process of selection, and selection can be biased even when the later analysis of a specific claim is correct.

Methodology on paper and methodology in practice

The second problem is consistency.

A Spanish study of the fact-checker Newtral showed that formal verification protocols are not always sufficiently precise or consistently implemented for results to be treated simply as the output of an almost mechanical process. The authors found that individual fact-checks contain a series of journalistic decisions: how to formulate the claim, which sources to consider, where to draw the boundary of context, and what level of evidence to require.

That is normal journalistic work. The problem arises when the result is presented as if it were something else — a neutral technical ruling.

Every fact-check contains at least three separate judgments. The first is what the actual claim was. The second is which data are relevant. The third is which label best summarizes the relationship between the claim and the evidence. Even a small shift in the first step can change the entire result. If we check a stronger version of a statement than the person actually made, we can correctly refute something they never said. If we check a narrower version, we can miss the point.

That is why the label “false” should never substitute for reading the argument.

A Slovenian example: Razkrinkavanje and the power of the label

In Slovenia, the most obvious example is the Razkrinkavanje.si project run by Oštro. Their publicly available methodology is, in several respects, a good illustration of how fact-checking should work: they explain what they select for checking, use categories such as “true,” “false,” “biased,” and “grey area,” and explicitly reserve the latter for cases that cannot be assessed unambiguously because of insufficient reliable data or relevant counterarguments. Oštro is also a verified member of the European EFCSN network.

This matters because it shows that serious fact-checking cannot operate with only two buttons, “truth” and “lie.” But even a good methodology does not eliminate the question of selection: the newsroom still decides which post to check, which part of a statement to formulate as the verifiable claim, and which sources receive the greatest evidentiary weight.

In 2022, Oštro became a partner in Meta's third-party fact-checking program. According to Oštro's own description, its task was to identify, review, and rate content, while Meta then attached warnings to rated content, reduced its distribution, and prevented advertising of such content. Oštro later specifically emphasized that it had no technical ability to delete posts or user profiles and that such measures were exclusively within Meta's control.

That distinction is essential. It is not accurate to say that a Slovenian fact-checker directly “deleted people from Facebook.” It is accurate to say that its editorial assessment could become an input signal in a platform system with concrete consequences for content visibility. Once that happens, fact-checking is no longer merely a journalistic article. It becomes part of the distribution infrastructure.

When a journalistic assessment gains technical power

Fact-checking becomes much more consequential when it ceases to remain an article and becomes infrastructure.

This was clearly visible in Meta's third-party fact-checking program. A fact-checker did not merely publish an analysis. Its rating could trigger warning labels, reduced distribution, and other platform consequences. In 2021, Meta explicitly explained that it reduced the reach of content rated false or misleading and that repeated sharing of such content could also affect the visibility of pages.

The nature of the problem changes here. If I publish an article saying you are wrong, we are participating in public debate. But if my assessment directly affects how many people can see your claim, my role is no longer merely journalistic. I become part of the distribution system.

In January 2025, Meta ended the independent third-party fact-checking program in the United States and began replacing it with Community Notes. The company justified the change partly by arguing that experts have their own biases and that, in its view, the system had intruded too far into legitimate political debate. That is Meta's own assessment and should not be confused with independent proof that all fact-checkers were politically biased. But it is an important acknowledgment by the company that had previously given their ratings direct platform power.

This reveals a broader problem. The question is not only whether a fact-checker judges an individual claim correctly. It is whether a private editorial judgment should automatically become a signal for algorithmic reduction of visibility.

When a “false claim” becomes an open question

The most instructive cases are not those in which a fact-check obviously gets a trivial fact wrong. More important are cases in which the boundary of knowledge shifts over time.

The origin of SARS-CoV-2 is a good example precisely because it is full of nuance.

In 2020, PolitiFact rated virologist Li-Meng Yan's claim that the virus had been created in a laboratory as “Pants on Fire.” Her claim was far more specific than the general laboratory-incident hypothesis: Yan spoke of an intentionally engineered virus, and her evidence was heavily criticized at the time. However, in May 2021 PolitiFact removed the fact-check from its active database and acknowledged that some of the sources behind the original categorical formulation about impossible laboratory manipulation had later become more contested. The claim was then described as unproven and disputed.

In February 2021, Facebook included among removable content the claim that COVID-19 was “man-made or manufactured.” Three months later, on May 26, 2021, it changed that policy and announced that it would no longer remove such claims because of new investigations into the virus's origins.

This does not mean it has been proven that SARS-CoV-2 originated in a laboratory. It has not. In June 2025, WHO assessed that the available evidence more strongly supported zoonotic spillover while emphasizing that, because of missing information, a laboratory incident could neither be confirmed nor excluded. The WHO Director-General put it even more directly: all hypotheses must remain on the table, including zoonotic origin and a laboratory incident.

The real lesson is therefore not “the banned theory turned out to be true.” That would be a new simplification. The lesson is more uncomfortable: an institution can treat something as false enough to remove even though the later state of the evidence shows that the broader version of the question was still open.

This reveals the danger of overly categorical labels. There is an enormous difference between “demonstrably false” and “we currently do not have enough evidence.” A system that erases the difference can create false certainty.

A fact-checker can be correct and still biased

This may be the most difficult part of the debate. Bias does not necessarily mean lying.

A fact-checker can correctly refute a selected claim, use good sources, and write an accurate article, while still being part of a biased information system if they systematically select only certain kinds of claims, a particular political camp, certain institutions, or certain topics.

It is similar to a photograph. A photograph can be completely authentic, but the frame is still a choice. What is absent from the photograph can be just as important to understanding the event as what is present.

That is why the demand “show me the mistake in the specific fact-check” is too narrow. Sometimes we must also ask: why was this sentence checked? Who chose the topic? How many comparable claims from the other side were not checked? Which experts were treated as authoritative? Who funds the organization? Are corrections as visible as the original judgments?

Such questions are not an attack on fact-checking. They are fact-checking the fact-checker.

A perception of bias is not always proof of actual bias

We must also be fair to the other side of the problem.

People often perceive a fact-checker as biased precisely when the assessment contradicts their own belief. Research shows that political identity strongly affects trust in the checker. An experiment in Argentina found that even a substantively correct judgment could shift the perceived ideological position of the fact-checker depending on which political side benefited from the result.

This means two phenomena can exist simultaneously.

First, fact-checking can genuinely be selective or institutionally biased. Second, audiences can perceive an unbiased or correct fact-check as biased merely because they dislike the outcome.

So the sentence “fact-checkers are biased” is not enough. We need evidence for the type of bias: selection, methodological, ideological, financial, editorial, or distributional.

Community Notes are not a flawless solution

When Meta or X transfer part of the responsibility to the community, centralized judgment is reduced, but the problem does not disappear.

Research on Community Notes shows that a displayed note can substantially reduce further sharing of a misleading post. One study found an approximately 46 percent reduction in subsequent reposts, while another found about a 61 percent reduction in diffusion after the note appeared. That is a concrete effect.

But speed is a problem. A note may appear only after most of the viral spread has already happened. On politically contentious topics, reaching consensus among people with different viewpoints can be difficult. A crowd-based system has different errors from a professional fact-checker, not necessarily fewer errors.

That is important: the alternative to a centralized arbiter is not necessarily a perfect decentralized marketplace of truth. Every model has its own architecture of bias.

Who checks the people who check?

The worst response to the problem of disinformation is replacing one form of blind trust with another.

If we are critical of a politician because they have an interest, we must also be critical of the media. If we are critical of the media, we must be critical of the fact-checker. If we are critical of the fact-checker, we must also be critical of the platform that uses the rating. And if the platform claims that the solution is a crowd-based system, we must examine its criteria too.

Trust should not be tied to the title of an institution. It should be tied to the process.

A good verification system therefore does not require untouchable arbiters. It requires visible primary sources, a clear distinction between fact and interpretation, disclosure of methodology and funding, a public history of corrections, a right of reply, and an appeals process. Above all, it must show uncertainty where uncertainty actually exists.

The label “we don't know” is sometimes more honest than “false.”

The European Union and the danger of criticizing the wrong thing

With the European Union too, it is important that we do not criticize something the system does not actually do.

Under the Digital Services Act, so-called trusted flaggers receive priority treatment when reporting allegedly illegal content. Platforms must process their reports as a priority. In Slovenia, this status is granted by AKOS, the Agency for Communication Networks and Services.

That is not the same as a right to decide what is true. The DSA does not grant them general authority to label political disagreement a “lie” and demand removal. Their formal role concerns allegedly illegal content.

But a legitimate question remains: how closely can the regulatory framework, platform rules, disinformation codes, fact-checkers, and civil-society organizations converge before it becomes almost impossible for the public to understand who made the decision in a particular case and according to which criterion?

Transparency is therefore more important than the label.

The greatest danger is not error, but automatic trust

A fact-checker can make a mistake. That in itself is not a scandal. Scientists, courts, journalists, and expert commissions make mistakes. The problem begins when the surrounding system creates a culture in which their assessments are no longer treated as arguments, but as the end of the debate.

Then the user no longer reads the evidence. They look at the badge.

A new form of epistemological laziness emerges. Instead of “I checked” we get “someone checked for me.” Instead of “what evidence?” we get “what label?”

Psychological research points to an additional trap: if a platform labels only some false content, unlabeled posts may gain an implicit appearance of credibility. This is the implied truth effect. A user may unconsciously conclude: if there is no warning here, it is probably fine.

But no system can check everything.

The absence of a label is therefore not proof of truth, just as the presence of a label is not a substitute for an argument.

Do not trust the fact-checker. Check the fact-checker.

The healthiest position is not: “fact-checkers always lie.” That would be precisely the same mistake we accuse them of — a categorical judgment broader than the evidence.

A better position is stricter: Do not trust a fact-checker because of their status. Check their work.

Look at the original statement. Check whether it has been represented correctly. Open the primary sources. Look at the date. Check whether consensus has been presented as proof. Look at what was omitted. Check the funding. Find the correction. Compare another independent source.

If the fact-check survives such scrutiny, it is useful. If it does not, the label has no special magical authority.

In an information society, we do not need a new priesthood handing out permits for truth. We need a better method of verification — including for those who put verification in their own name.

And that is why the real question is not: Who decides what is true? The real question is:

Who checked the claim, with what evidence, by what method — and can we check them too?