John Poindexter: The Fed's Uber Geek

Is retired Admiral John Poindexter (AJP) the best software architect alive today? I think he just might be, and I'm not sure whether to feel warm-and-fuzzy about that - or to be very, very afraid. That's because Poindexter, a convicted felon with a criminal record of lying to congress, is the federal government's "Uber-Geek." Nerd in chief. The individual most responsible for the conception, design and execution of the DHS's massive security technology apparatus. At least the data mining and analysis piece, which is the core functionality of the new system. His convictions, I should note, were modified on appeal.

AJP has been working on this stuff for years. And I know that to be a fact. I first met the Admiral in the late 80s - when he was back in civies after Iran-Contra. He was a developer at the time, and worked with DESQview/X, a multi-tasking, windowing environment provided by my employer. He showed up at the annual developer's conference one year, and joined the company's trade-show crew and other developers on the floor of Network World (might have been InterOp) on at least one occasion that I remember. I also remember him hanging with Wayne Ratliff and Bob Byers, of dBASE fame, who shared his passion for software design and database architecture. Even then, there were lots of animated conversations around collecting data from disparate sources on multi-platform networks; buffering and caching that information in novel constructs, while providing multi-layer hooks and handles so that external software agents could organize, optimize and process the aggregated data to achieve certain, pre-defined goals. Heady stuff, at the time. There was a lot going on in the data base developer universe in those days, Brian Russell and Mitch McConnell had finished Clipper 5.0. Ratliff & Byers were collaborating on new stuff, and The Admiral was right in the middle of all the action.

Poindexter was the mind behind the Total Information Access (TIA) system first proposed after 9/11. Congress was so intimidated by the reach and constitutional implications of that model they cut off funding and told the program to go away in 2003. It didn't, of course. Now, TIA has returned on steroids - and has been rebranded as ADVISE - for Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement. Parse that. It makes for some interesting disambiguation. The Christian Science Monitor covered the story here.

ADVISE is capable of cross-matching material from websites and blog posts to government records and personal data. Techies, ponder this: The Feds can take ten-terabyte gulps of data and process it in under a minute. As good as that may be for advancing scientific research, it does not bode well for personal privacy.

As I understand it, massively scalable, grid- or blade-based computing architectures based on commodity components connected by 10Gb Ethernet networks and Core XGE Ethernet-based backbone switches are delivering over one terabyte/minute table scanning rates in a variety of available Data Warehouse Appliance configurations. Another model, "share-nothing" architecture, is the key to the MIT Media Lab's new petabyte storage, terabit I/O bandwidth "Human Speechome" project. That is one of the largest and highest performing commercial systems that I am aware of. And you know the Feds are way ahead of that curve.

And now, the feds have a great Beta site for all of their machinations: Singapore. Wired Magazine recently reported on the re-birth of TIA/ADVISE and the work of AJP in rolling-out the tightly controlled country's internal surveillance and data mining program. Dubbed Risk Assessment and Horizon Scanning (RAHS), the new system is being deployed by Poindexter protegees. Singapore is widely expected to fine-tune the system, work out any bugs, then sell it back to us. It's just so convenient that I can't avoid the conclusion that the old "nod-and-wink" is at work to circumvent the will of the Congress of the United States and the opinions of our citizens. Just peachy.

A worthy data mining and privacy guru, Jeff Jonas of IBM (check out his fine blog here), adds this humorous take: "Data Mining, noun, 1. Torturing data until it confesses ... and if you torture it enough, it will confess to anything." Therein lies the conundrum for privacy advocates. I can guarantee that outsourcing our systems development and beta testing to entities that are beyond our authority and public oversight will only exacerbate that conflict. But then again, that's how AJP and his crew have always worked.