Surveillance and privacy
Discussion of issues related to liberty and privacy, and especially how they are affected by and interrelated with data management and analytic technologies. Related subjects include:
Petabyte-scale data management
Privacy, censorship, and freedom (in The Monash Report)
Political issues around big tech companies
The technology industry has an increasingly complex relationship to government and politics, most importantly in three areas:
- Privacy and surveillance.
- Censorship.
- Antitrust, general economic regulation, and other competition management.
Here’s some of what I think about that, plus links to a lot more.
1. For a long time, I’ve maintained:
- Privacy and surveillance are very big deals.
- Ultimately, they cannot be handled effectively without direct regulation of specific permitted and forbidden uses of data.
The first point is now widely accepted. The second unfortunately is not; laws and regulations generally state who may or may not record, keep or decrypt particular kinds of data, rather than what particular uses they may make of it.
2. Another threat to freedom has arisen as big as that from privacy/surveillance: a many-fronts push for censorship. It would ultimately be calamitous for free countries to agree that the threat of “Fake News” and other dangerous online partisanship justifies general censorship, by governments or “platform” tech companies as the case may be, yet that is exactly the path we seem to be on.
Fortunately, there are less dangerous ways to address the same challenges. I expect to make as much fuss about this issue in the upcoming decade as I have about privacy/surveillance over the past one.
| Categories: Surveillance and privacy | 4 Comments |
New legal limits on surveillance in the US
The United States has new legal limits on electronic surveillance, both in one specific way and — more important — in prevailing judicial theory. This falls far short of the protections we ultimately need, but it’s a welcome development even so.
The recent Supreme Court case Carpenter v. United States is a big deal. Let me start by saying:
- Most fundamentally, the Carpenter decision was based on and implicitly reaffirms the Katz test.* This is good.
- The Carpenter decision undermines the third-party doctrine.** This is great. Strict adherence to the third-party doctrine would eventually have given the government unlimited rights of Orwellian surveillance.
- The Carpenter decision suggests the Court has adopted an equilibrium-adjustment approach to Fourth Amendment jurisprudence.
- The “equilibrium” being maintained here is the balance between governmental rights to intrude on privacy and citizens’ rights not to be intruded on.
- e., equilibrium-adjustment is a commitment to maintaining approximately the same level of liberty (with respect to surveillance) we’ve had all along.
- I got the equilibrium-adjustment point from Eugene Volokh’s excellent overview of the Carpenter decision.
*The Katz test basically says that that an individual’s right to privacy is whatever society regards as a reasonable expectation of privacy at that time.
**The third-party doctrine basically says that any information of yours given voluntarily to a third party isn’t private. This includes transactional information such as purchases or telephone call detail records (CDRs)
Key specifics include: Read more
| Categories: GIS and geospatial, Surveillance and privacy | 1 Comment |
Some things I think about politics
When one tries to think comprehensively about politics these days, it quickly gets overwhelming. But I think I’ve got some pieces of the puzzle figured out. Here they are in extremely summarized form. I’ll flesh them out later as seems to make sense.
1. Most of what people are saying about modern tribalism is correct. But partisanship is not as absolute as some fear. In particular:
- There are populist concerns on the right and left alike.
- Partisans of all sides can be concerned about privacy, surveillance and government overreach.
2. The threat from Trump and his Republican enablers is indeed as bad as people fear. He’s a major danger to do terrible, irreversible harm to the US and the rest of the world. To date the irreversible damage hasn’t been all that terrible, but if Trump and his enablers are given enough time, the oldest modern democracy will be no more.
All common interests notwithstanding, beating Trump’s supporters at the polls is of paramount importance.
3. I agree with those who claim that many of our problems stem from the shredding of trust. But few people seem to realize just how many different aspects of “trust” there are, nor how many degrees there can be of trustworthiness. It’s not just a binary choice between “honest servant of the people” and “lying, cheating crook”.
These observations have strong analogies in IT. What does it mean for a system to be “reliable” or to produce “accurate” results? There are many possible answers, each reasonable in different contexts.
| Categories: Public policy, Surveillance and privacy | 3 Comments |
Politics can be overwhelming
Like many people, I’ve been shocked and saddened by recent political developments. What I’ve done about it includes (but is not limited to):
- Vented, ranted and so on. That’s somewhat therapeutic, and also let me engage the other side and try to understand a little better how they think.
- Tried to understand what’s happening. I probably have had more available time to do that than most people. I also have a variety of relevant experiences to bring to bear.
- Neglected my work somewhat while doing all that. This neglect has now stopped. After all, the future is quite uncertain, so we should probably work hard in the present while business is still good.
- Written up some of what I’ve figured out. Of course. That’s what I do. But it’s only “some”, because … well, the entirety of politics is overwhelming.
- Tried to find specific, actionable ways to help. Stay tuned for more on that part.
As for those writings: Read more
| Categories: Public policy, Surveillance and privacy | 2 Comments |
The chaotic politics of privacy
Almost nobody pays attention to the real issues in privacy and surveillance. That’s gotten only slightly better over the decade that I’ve written about the subject. But the problems with privacy/surveillance politics run yet deeper than that.
Worldwide
The politics of privacy and surveillance are confused, in many countries around the world. This is hardly surprising. After all:
- Privacy involves complex technological issues. Few governments understand those well.
- Privacy also involves complex business issues. Few governments understand those well either.
- Citizen understanding of these issues is no better.
Technical cluelessness isn’t the only problem. Privacy issues are commonly framed in terms of civil liberties, national security, law enforcement and/or general national sovereignty. And these categories are inherently confusing, in that:
- Opinions about them often cross standard partisan lines.
- Different countries take very different approaches, especially in the “civil liberties” area.
- These categories are rife with questionably-founded fears, such as supposed threats from terrorism, child pornographers, or “foreign interference”.
Data sovereignty regulations — which are quite a big part of privacy law — get their own extra bit of confusion, because of the various purposes they can serve. Chief among these are: Read more
| Categories: Surveillance and privacy | 2 Comments |
Notes on artificial intelligence, December 2017
Most of my comments about artificial intelligence in December, 2015 still hold true. But there are a few points I’d like to add, reiterate or amplify.
1. As I wrote back then in a post about the connection between machine learning and the rest of AI,
It is my opinion that most things called “intelligence” — natural and artificial alike — have a great deal to do with pattern recognition and response.
2. Accordingly, it can be reasonable to equate machine learning and AI.
- AI based on machine learning frequently works, on more than a toy level. (Examples: Various projects by Google)
- AI based on knowledge representation usually doesn’t. (Examples: IBM Watson, 1980s expert systems)
- “AI” can be the sexier marketing or fund-raising term.
3. Similarly, it can be reasonable to equate AI and pattern recognition. Glitzy applications of AI include:
- Understanding or translation of language (written or spoken as the case may be).
- Machine vision or autonomous vehicles.
- Facial recognition.
- Disease diagnosis via radiology interpretation.
4. The importance of AI and of recent AI advances differs greatly according to application or data category. Read more
| Categories: Cloud computing, Predictive modeling and advanced analytics, Public policy, Surveillance and privacy | 4 Comments |
Notes on data security
1. In June I wrote about burgeoning interest in data security. I’d now like to add:
- Even more than I previously thought, demand seems to be driven largely by issues of regulatory compliance.
- In an exception to that general rule, many enterprise have vague mandates for data encryption.
- In awkward contradiction to that general rule, there’s a general sense that it’s just security’s “turn” to be a differentiating feature, since various other “enterprise” needs are already being well-addressed.
We can reconcile these anecdata pretty well if we postulate that:
- Enterprises generally agree that data security is an important need.
- Exactly how they meet this need depends upon what regulators choose to require.
2. My current impressions of the legal privacy vs. surveillance tradeoffs are basically: Read more
| Categories: Data warehousing, Databricks, Spark and BDAS, EAI, EII, ETL, ELT, ETLT, Hadoop, Surveillance and privacy | Leave a Comment |
Light-touch managed services
Cloudera recently introduced Cloudera Altus, a Hadoop-in-the-cloud offering with an interesting processing model:
- Altus manages jobs for you.
- But you actually run them on your own cluster, and so you never have to put your data under Altus’ control.
Thus, you avoid a potential security risk (shipping your data to Cloudera’s service). I’ve tentatively named this strategy light-touch managed services, and am interested in exploring how broadly applicable it might or might not be.
For light-touch to be a good approach, there should be (sufficiently) little downside in performance, reliability and so on from having your service not actually control the data. That assumption is trivially satisfied in the case of Cloudera Altus, because it’s not an ordinary kind of app; rather, its whole function is to improve the job-running part of your stack. Most kinds of apps, however, want to operate on your data directly. For those, it is more challenging to meet acceptable SLAs (Service-Level Agreements) on a light-touch basis.
Let’s back up and consider what “light-touch” for data-interacting apps (i.e., almost all apps) would actually mean. The basics are: Read more
| Categories: Cloud computing, Cloudera, Data warehousing, EAI, EII, ETL, ELT, ETLT, Hadoop, Software as a Service (SaaS), Surveillance and privacy | 3 Comments |
There’s no escape from politics now
The United States and consequently much of the world are in political uproar. Much of that is about very general and vital issues such as war, peace or the treatment of women. But quite a lot of it is to some extent tech-industry-specific. The purpose of this post is outline how and why that is.
For example:
- There’s a worldwide backlash against “elites” — and tech industry folks are perceived as members of those elites.
- That perception contains a lot of truth, and not just in terms of culture/education/geography. Indeed, it may even be a bit understated, because trends commonly blamed on “trade” or “globalization” often have their roots in technological advances.
- There’s a worldwide trend towards authoritarianism. Surveillance/ privacy and censorship issues are strongly relevant to that trend.
- Social media companies are up to their neck in political considerations.
Because they involve grave threats to liberty, I see surveillance/privacy as the biggest technology-specific policy issues in the United States. (In other countries, technology-driven censorship might loom larger yet.) My views on privacy and surveillance have long been:
- Fixing the legal frameworks around information use is a difficult and necessary job. The tech community should be helping more than it is.
- Until those legal frameworks are indeed cleaned up, the only responsible alternative is to foot-drag on data collection, on data retention, and on the provision of data to governmental agencies.
Given the recent election of a US president with strong authoritarian tendencies, that foot-dragging is much more important than it was before.
Other important areas of technology/policy overlap include: Read more
| Categories: Public policy, Surveillance and privacy | 5 Comments |
Politics and policy in the age of Trump
The United States presidency was recently assumed by an Orwellian lunatic.* Sadly, this is not an exaggeration. The dangers — both of authoritarianism and of general mis-governance — are massive. Everybody needs in some way to respond.
*”Orwellian lunatic” is by no means an oxymoron. Indeed, many of the most successful tyrants in modern history have been delusional; notable examples include Hitler, Stalin, Mao and, more recently, Erdogan. (By way of contrast, I view most other Soviet/Russian leaders and most jumped-up-colonel coup leaders as having been basically sane.)
There are many candidates for what to focus on, including:
- Technology-specific issues — e.g. privacy/surveillance, network neutrality, etc.
- Issues in which technology plays a large role — e.g. economic changes that affect many people’s employment possibilities.
- Subjects that may not be tech-specific, but are certainly of great importance. The list of candidates here is almost endless, such as health care, denigration of women, maltreatment of immigrants, or the possible breakdown of the whole international order.
But please don’t just go on with your life and leave the politics to others. Those “others” you’d like to rely on haven’t been doing a very good job.
What I’ve chosen to do personally includes: Read more
| Categories: About this blog, Public policy, Surveillance and privacy | 9 Comments |