How to beat “fake news”
Most observers hold several or all of the views:
- “Fake news” and the like are severe problems.
- Algorithmic solutions have not worked well to date.
- Neither have manual ones.
- Trusting governments to censor is a bad idea.
- In light of the previous points, trusting large social media corporations to censor is a bad idea too.
- Educating consumers to evaluate news and opinions accurately would be … difficult.
And further:
- Whatever you think of the job traditional journalistic organizations previously did as news arbiters, they can’t do it as well anymore, for a variety of economic, structural and societal reasons.
But despite all those difficulties, I also believe that a good solution to news/opinion filtering is feasible; it just can’t be as simple as everybody would like.
1. When people think about these problems, they’re probably most focused on social media platforms such as Facebook, YouTube or Twitter. But before getting to those, let’s consider the simpler case of search engines. In essence, what search engines do is:
- Assign a relevance score to the relationship between a site (or particular page) and your query.
- Assign a quality score to a site.
- Combine those two scores into an overall ranking, and serve up results accordingly.
How well does this work? I’d say that search engines:
- Are good at directing you to information that is generally related to what you want to know. This is the technological core of what they do.
- Are good at shielding you from the worst cheating/spammer/hacker sites. That’s also a major technical focus for them.
- Are poor but not horrible at distinguishing between good and bad sources of information, opinion or advice. Generally, they do this via some kind of popularity contest, whether via Google’s venerable PageRank or by more directly observing which sites users seem to go to and stay at.
- Don’t even try to filter sites according to leanings such as political bias.
Lessons from that start:
- Huge technology companies can actually do pretty well at the parts of the problem that technology alone can solve.
- A lot of the challenge boils down to adversarial information retrieval, where the adversaries range from somewhat honest polemicists or hucksters to completely awful hackers and spammers.
2. When defending against bad actors, scale helps a lot. In my favorite example:
- A significant fraction of all the world’s email goes through Gmail.
- Thus, it is very hard for email spam blasts to escape Google’s honeypots.
- Informed by those honeypots, Google does what in my opinion is a very good job of fighting spam.
Similarly, as the publisher of multiple blogs, I can tell you that much the same is true of WordPress’ Akismet’s fight against spam comments. Akismet isn’t perfect; indeed, I’ve stopped adding new content to the blog where this post would fit best — Text Technologies – because of a multi-year spam attack. But on the whole Akismet works very well.
Thus, in contradiction to many observers, I believe that the huge scale of social media companies is NOT the root of the problem.
3. Of course, concern is really focused on social media, and especially on the concern that people communicate things they (supposedly) shouldn’t, where:
- “Communicating” includes words, pictures, videos, etc.
- “Shouldn’t” covers outright lies, great factual distortions, hate speech … or just opinions that the would-be censor doesn’t think should be spread.
And even if you don’t worry so much about those problems, some kind of censorship, filtering or gatekeeping is inevitable anyway, simply because there’s vastly more information in the world than any one person can consume.
So what are the main options for censorship and other gatekeeping? My opinions start:
- Having governments be in charge of censorship is a terrible idea.
- Having large, non-journalist corporations be directly in charge of censorship is also a bad idea, because ultimately they’ll just succumb to government or other political pressure.
- The traditional modern gatekeepers are journalistic organizations, who both deserved and received trust that they’d do the job responsibly. But that model no longer suffices in its old form, for several sets of reasons:
- New requirements. Traditional journalistic gatekeeping boils down to organizations vetting the content they themselves produce. It doesn’t work nearly as well for third-party content, worthy efforts such as fact-checking columns notwithstanding.
- Trust. Walter Cronkite is long dead; journalists aren’t nearly as trusted as they used to be.
- Deliberate bias. Opinion and bias are now part of many “news” organizations’ business models, to a much greater extent than they were a few decades in the past.
- Money. Most journalistic organizations have much slimmer news budgets than they had at their peak.
4. So if we need gatekeeping, and no natural kind of gatekeeper can on its own be effective or safe, what’s left? In simplest terms, we need gatekeeping by (technological) committee. Mainly, what I propose comprises:
- Different kinds of gatekeeper for different aspects of the problem, including at a minimum:
- Human-led filters to deal with various issues in credibility and bias.
- Technology-led filters to deal with pure fakery and false provenance.
- Further filtering of the kinds that would be needed even in a more benign world.
- Multiple choices for at least the human-led filters.
- Good, simple (!) user interfaces for combining those filters’ results.
Above all, people must be able to choose their own censors.
5. What I envision for the “human-led filters to deal with various issues in credibility and bias” is something like:
- An organization maintains slowly-changing whitelists and blacklists of information sources.
- The same organization fact-checks or other vets specific stories, claims and content in near-real-time.
- The results scale to other stories, claims and content via very-rapidly-retrained machine learning models, whether those are based on a single gatekeeper’s hand editing or, more likely, on multiple hand-edited training sets and other collaborative inputs at once.
Here an “organization” can be anything trusted by enough people to be economically viable, for example:
- An offshoot of an existing journalistic organization.
- An offshoot of an existing political party or advocacy organization.
- A successfully started-up new entity.
Obviously, there would be business issues, notably:
- Costs. Who pays, in an economy where news is commonly “free”?
- Chicken-egg adoption. Which gets developed first: Human-led filtering services that can’t yet be integrated into actual social media filtering, or technology to integrate human-led filtering services that don’t yet exist?
But given the importance and visibility of the problem, optimism about solving the business issues is appropriate. The hardest part is the technology itself. Can machine learning models be retrained on a sub-hour or even sub-minute basis? Sure. That’s been confirmed many times. But what I’m suggesting is a pretty complex case, with global scale, intermediate results passed among organizations, with plenty of adversarial elements, all done at very high speed.
That is not yet a solved problem. But it certainly seems solvable. Further, it’s a problem that must be solved, lest liberal democracy be as doomed as some people fear it actually is.
Related links
- I wrote a bit about adversarial analytics in May, 2016.
- I outlined my views about the “War(s) on Truth” in February, 2018.
- Earlier this month, Cory Doctorow offered a hard-hitting column on the dangers of expecting internet companies to do our censorship for us.
- More sedately but with more explanation, so did Will Oremus.
Comments
9 Responses to “How to beat “fake news””
Leave a Reply
[…] there are less dangerous ways to address the same challenges. I expect to make as much fuss about this issue in the upcoming […]
Fantastic and Knowledgeable Blog…Thanks for Sharing!!!
Best photographer in Allahabad
Demon Slayer Merch
How to beat "fake news" | DBMS 2 : DataBase Management System Services
An intriguing discussion is worth comment. I do think that you should publish more about this topic,
it might not be a taboo subject but typically folks don’t discuss these topics.
To the next! All the best!!
From designing simple websites to developing
robust software applications or crafting the most effective digital marketing strategy with SEO, AdWords and social media at the core,
we are the perfect web design company in Srilanka for businesses of all sizes.
You’ve provided so much clarity. Much appreciated!
Wonderful write-up! Your explanation on topic was spot on. Have you considered exploring another aspect?
Thanks for sharing this article! Do you think
software solutions will simplify this process in the near future?
I have long spoken out for free speech. I did not always do it in the best, most politically correct manner, and I was often misunderstood, or I set myself up to be misunderstood from that time onward.
Other times I was understood correctly and it pissed people off because they had different convictions.
But one form of cowardice, bigotry, and oppression is for a group of people to decide what others are allowed to say, hear, write, read, contemplate, and believe. That is not a violation of fake “rights” or those that overgrown adolescents claim without honest reason.
We’ve seen discrimination and lousy excuses for discrimination given as though the excuses were somehow intelligent and honest when they clearly were not. We’ve seen a vicious and unjust cancel culture, a restriction of career growth and office politics and cowardly defamation all to asuage the bigotry of some very inept and unethical and self-righteous hypocrites in management who could not manage their way out of a paper bag let alone run a company without running it into the ground and seeing it taken over by a company like Computer Associates. It would seem the engineers had a good reason to walk out.
And sadly, a culture that continued to live in the hearts of people who remembered the good things about it, ceased to live in the land of reality.
We boasted about our technical superiority to Oracle but had no evidence to support that. Kick the plug out of the wall on a computer running Ingres and a computer running Oracle, and Oracle returns to live while Ingres needed someone to detangle things. I know. I detangled them countless times.
And although Oracle was a diverse company in every sense, it did not have the bigoted idiocy that seemed to seeth toward anyone who had any leaning other than that which was wildly leftist.
And yet, Ingres had many good, honorable, friendly, committed people of every religion, every political leaning, gay, straight, old, young, and most of them were not cruel and back-stabbing cowards. Most were sincere, kind, faithful friends and coworkers, and I loved them.
If I saw the heart of Ingres return and the quality of technology return, and if I saw company become the company I fell in love with in the 1990s, I would be intensely thankful and supportive to the core of my being. And yet, to be frank, it would seem Oracle is not perfect, but it is the beautiful company I also learned to love in the 1990s.
For about a decade, I hated changing companies a little. I wanted to see Oracle become great. It didn’t. It died. I did not want to move to a competitor, but I did. And I was glad I did. My immediate coworkers also left.
At the time, we had been moved from a great, wonderful director who we loved–a lady who was a bit like a loving mother-hen of sorts caring for her employees. And we were moved from her to a gentleman who was good in many ways–decisive, but seemed inclined to fire his way out of problems rather than face them, understand them, and solve them with competence. Shortly after, we were moved to another director who was a rather hard hearted woman with about a third the wisdom of that gentleman. And a manager was hired who knew little to nothing about our department.
Instantly, we were put on a performance plan. The same day, I received a supporting star award for going beyond our duties.
The team was going to walk off the campus and never look back leaving the manager and director completely without a staff. And perhaps we should have. They were stupid, inept, unethical, heartless, cowardly, arrogant, and clueless as managers. And it seemed Ingres was not very good at hiring managers very intelligently, though some were indeed good. I don’t know how the good ones got selected, but I was thankful for every one of them.
I encouraged our team to stay and face the challenge. It was unjust. It was stupid. The manager and director were both being compete asses. But their actions presented a challenge that I felt each person could grow from including myself.
Wouldn’t it be better to leave to go TOWARD a goal rather than AWAY from an unpleasant situation? Wouldn’t it be better to try and succeed than to give up? What if we took up the challenge, looked at the specific requirements for staying at Ingres, met every requirement and exceeded it with excellence first, and then waited until we had our ideal job waiting for us to go to?
And that’s what we did. Each member on the team met the requirements and came off the fake performance plan. We faced the challenge and we overcame.
And the day we were congratulated and taken off the plan, each and every one of us had our ideal job waiting for us, and we submitted our resignation.
The director left and her new company she went to collapsed. The manager remained, but the company was sold to Computer Associates and the engineers walked. The CEO of CA said the engineers were sour sports and nobody would want them. Yet I saw my old friends showing up at Oracle soon after. And an airplane flew overhead inviting ex-Ingres employees to come and work for Sybase, another great database company.
So, what happened after that? I loved Oracle. Some burned out because the work was intense. I went to grad school at Stanford part time and studied Advanced Systems and Databases. And the training I received was awesome. After a couple years as a production system admin and member of the security response team, I moved to a new role with a team to create the Enterprise Systems Center–basically world class database and systems experts doing applied research in performance, scalability and such.
And I didn’t want to leave Oracle. But PeopleSoft kept pulling me to work for them, and they were only five minutes from my home. So I eventually caved and loved PeopleSoft as much as I loved Oracle. I became part of a software architecture team developing the esupport or automated diagnostics system based on Motive extending it to multi-tiered environments–actually my friend Chad did that while I integrated it with our CRM system. I helped develop the training with a lady on the team, and our team went to New Orleans to present it to customers at a big PeopleSoft event.
A startup company called Clickmarks then contacted me and hired me to be their Chief Systems Architect and Director of IT and Operations where I built their IT Department and helped build and manage their QA and Support departments. Our first massive customer was Vodafone Japan and that expanded out to other places in the world.
So, had I remained at Ingres, what opportunities would I have missed?
Sometimes difficulties can bring about great results in the long run.
And yet here I am hoping and praying for Ingres to have another great revival and great success.
Companies provide a place for people to work and satisfy their need to leave a legacy benefiting other people. I hope and pray to see Ingres do well.