Palantir for beginners or which circle of hell is yours

Among the clients of Palantir Foundry is the pharmaceutical giant Merck & Co, which, in partnership with Black & Veatch, took part in the construction of the notorious Fort Detrick and numerous biolabs – including the infamous Wuhan Institute of Virology. And that’s not all.

Palantir’s first product, developed since 2003 and released in 2008, was Gotham, named after the noir city from DC’s Batman comics. As is well known, the only superpowers of that hero were money and intelligence—he fought crime thanks to powerful analytical abilities and the fact that his command center had a vast number of powerful computers connected to various databases. This allowed for complex analysis, the uncovering of criminal schemes and suspects, predicting vulnerable spots that criminals might target and how they might do so, etc. This enabled one person to wage a war against an entire city’s mafia on equal footing.

FULFILLMENT OF A TEENAGE DREAM OR HOW TO BECOME BATMAN

Gotham was conceived as an intelligence-analytical system of the same type – a kind of fulfillment of a geek’s teenage dream about what it would be like to be Batman and see all the opponent’s moves in advance. The creators of Gotham, however, were not pioneers in this field – the foundational technology of this kind appeared in parallel with other expert systems as early as the 1970s. The first among them was the famous ECHELON from the NSA, created in the 1970s for global monitoring of intelligence information – and by analogy with it, a portrait of the capabilities of the Palantir system can be composed.

ECHELON was the first (and one of the most expensive in the world) system of global signals intelligence, which was developed over more than 10 years until the mid-1970s. It was developed for intercepting and analyzing electronic communications (radio, telephone, fax, and later – computer networks) and was used by the countries of the Five Eyes alliance (USA, United Kingdom, Canada, Australia, New Zealand). Its core consisted of a network of radio interception stations around the world. ECHELON collected data from numerous sources: satellites – jamming and intercepting satellite communication signals, ground antennas – intercepting radio transmissions, telephone conversations, faxes, underwater cables – wiretapping with the help of special devices from the NSA and the U.S. Navy, from the 1980s onward – internet communications, filtering data through network nodes.

WIRETAPPING VIA SATELLITES

One of the key elements of the system were the Keyhole and Rhyolite satellites, which could register radio signals and even eavesdrop on telephone conversations. The intercepted data were processed through an automatic analysis system, which operated on a network of, by today’s standards, old but very powerful mainframes and included subsystems for extracting keywords (for example, phrases such as “kill,” “bomb,” “terrorist attack,” “government,” “president,” etc.), analyzing textual messages (it monitored telegrams and emails, as well as early network bulletins), and recognizing voices to decrypt telephone conversations and identify callers. All the data were entered into a large database over which the expert system could conduct analysis and discover interconnection

This worked as follows – for example, in a telephone conversation, the word bomb was mentioned. After that, the voice would be compared with samples from the database and identified, say, as the voice of a terrorist from the IRA. Then the data would be forwarded to the British, and simultaneously the tapping of all nearby phones would be initiated. At the same time, computers that could be connected to that person would be searched for: emails that had been sent from them. It would also be checked whether the suspect had used the postal service and sent packages, and so on.

INTELLIGENCE PROGRAMS FOR DATA PROCESSING

In the 1990s, the system was expanded to work with banks – it now became possible to additionally check all accounts and credit cards of that person, to track who sent them money and when, where they spent it and on what, etc. Numerous scandals have been linked to the ECHELON system, since its satellites eavesdropped on military and diplomatic communication channels, as well as mobile conversations in different countries, leading to suspicions that it was being used as part of the competitive struggle of American corporations and to monitor NATO allies. The matter escalated to the point that the European Parliament launched an investigation in 1999, suspecting that ECHELON was used for industrial espionage against European companies. Details of the system’s operation were disclosed only in 2001, and at the same time, its new version was already operational – PRISM (exposed by Edward Snowden in 2013 – it turned out the NSA was also monitoring every American citizen). There are also alternative projects such as XKeyscore – for analyzing users’ internet activity, or Tempora (GCHQ) – the British version of internet surveillance. Since the 1990s, more global projects have been developed for the CIA and NSA, such as ThinThread – an intelligence program for big data analysis (the project was frozen after September 11), and Trailblazer (discontinued in the mid-2000s).

LAWSUIT AGAINST PALANTIR: THEFT OF CODE AND ALGORITHMS

The second pillar of the system is the visualization and presentation of information, and in this field too, Gotham was not a pioneer. As early as the 1990s, the United Kingdom saw the appearance of the program i2 Analyst’s Notebook – the first serious system for visual analysis of links between objects (used by law enforcement agencies). This is a close analog of the famous “crazy board” – the conspiracy theorist’s board, with photographs, notes, and a multitude of colorful strings pinned in place, well known to any fan of detective series. Similar functionalities are found in LexisNexis, and IBM i2 (from the 2000s) – corporate systems for the analysis of legal and financial information and the graphic presentation of interconnections. Interestingly, in 2011, the company i2 Inc. filed a lawsuit in federal court against Palantir, accusing it of fraud, conspiracy, and copyright infringement regarding graph algorithms. It turned out that Shyam Sankar, Director of Business Development at Palantir, had used a private detective agency and industrial espionage to steal i2’s source code and algorithms.

Naturally, geoinformation systems (GIS), created as early as the 1980s, were also involved. For example, ESRI ArcGIS – a program for cartographic analytics that has been used in special services for over 30 years. Google Earth Enterprise (2004) also allows working with cartographic data for intelligence purposes. All these tools worked with specific types of data, but they were not integrated into unified ecosystems. Everything changed in the mid-2000s, when new specialties emerged: data mining, data engineering, big data processing, and software products designed for these purposes.

ANALYTICS WITH A MISSION

One of the starting points on the path to the creation of Gotham was the PayPal Fraud Detection system (2001), developed by the founders of PayPal, which was necessary for detecting potentially fraudulent, corrupt, or “gray area” transactions. Its graph analysis mechanism was almost entirely transferred into Gotham. In 2004, their first integrated universal analytics platform appeared, which included all of the aforementioned: subsystems for extracting different types of information, cleaning that data, uploading it into a database, creating an ontology based on it, and algorithms that performed calculations on the ontology—in other words, conducted analysis and provided answers to defined questions. It was called Palantir Foundry, and was later divided into civilian and public versions—namely, Foundry and the state intelligence platform Gotham, which is used by police, the military, and security agencies.

The primary purpose of Gotham is to monitor and analyze military operations (it is used in this capacity in Ukraine) or for police profiling and the fight against organized crime (El Salvador, partially Mexico, the U.S.). Naturally, Gotham can be used for any type of analytics in which the task is to find non-trivial connections in vast amounts of raw, complex data (maps, photographs, phone calls, radio intercepts, banking transactions, and even newspaper clippings).

DIGITAL PROFILES OF THOUSANDS OF PEOPLE

Gotham’s operation is divided into the following phases. Aggregation of information. Gotham collects and aggregates data from numerous sources to which it is connected, starting from already linked existing databases to posts on social networks, surveillance camera footage, financial transactions, radio and phone conversations, and even paper intelligence reports (from which it recognizes and extracts key information—this is where the built-in standard neural network of the LLM type comes into play, ideal for working with text). After being loaded, the data undergo normalization and, if necessary, enrichment.

Search for connections. In the second phase, Gotham creates its own crazy board—connecting all incoming information through hyperlinks, forming a complete ontology. For example, it recognizes the face of every person appearing in a video, retrieves the entire police database (if available), their complete credit history, adds GIS geolocations showing where and when they were, reviews and records all their phone calls, emails, and all posts on their VKontakte wall. This results in a complete digital portrait of a person for, say, the past month—and this can be done for thousands of people. Then the system starts “pulling the strings” from one to another: who spoke to whom, who spoke about whom, who transferred money to whom, who met with whom. This way, a massive ontology is created that thoroughly describes every step within a certain time period—for example, an entire urban neighborhood.

Naturally, something like this requires extremely powerful hardware, and without the development of server technologies and Big Data algorithms in the 2010s, it would not have been possible. All information is stored in graph databases—essentially, electronic versions of a giant conspiracy theorist board with millions of hyperlinks.

SIMULATION OF POSSIBLE MOVES

Prediction and analysis. In the third phase, Gotham, using machine learning technologies—including both traditional expert systems and various types of LLM models—responds to user queries. For example: who committed a crime, when and with whom, who paid for it, with what funds, who stole the vehicle used to transport the body, where it was taken, and so on.

Moreover, Gotham’s power lies not only in its ability to determine what happened post factum, but also (for instance, in the case of military intelligence services) to make meaningful forecasts based on the constructed model network of actors—for example, on what day and at what hour an offensive will begin, where, with what forces, and what its objective will be. Naturally, Gotham can handle an entire spectrum of derivative tasks—tracking and predicting movement (who will be where, when, and with whom), semantic analysis (intelligence gathering from open sources, extracting and compiling textual information, searching for hidden meanings and interconnections between publications).

Gotham is also capable of chronological analysis and reconstruction: for instance, it can recreate not only causal and logical relationships between events, but also their chronological order. At the final level of prediction, Gotham creates an operational model of all actors, which allows, for example, pre-simulation of all possible moves by terrorists targeting a particular site.

It is clear that such a level of prediction requires three things: massive computing power, highly qualified personnel, and, most importantly, the need to provide the system with full access to all state databases and information collection sources.

BILLIONS OF EUROS IN COMPLETE SECRECY

The effectiveness of Gotham grows exponentially with the inclusion of an increasing number of data sources—in theory, it can provide total control over society, provided it has access to everything from customs and postal services, to mobile operators, banks, surveillance cameras, police reports, and more. With such access, state monitoring of individual behavior becomes possible.

It is no surprise that the U.S. government pays Palantir billions of dollars under conditions of complete secrecy when it comes to actual cases of usage. Because it can be used not only for benevolent purposes—it is also possible, if desired, to predict, for example, when and where a secret convoy with European diplomats will pass and direct a few jihadists toward it who somehow managed to bypass all borders and checks.

Palantir Gotham is used by counterterrorism analysts in the offices of the U.S. Intelligence Community and the U.S. Department of Defense, by financial fraud investigators at the Oversight and Transparency Board, and by cyber analysts at the Information Warfare Monitor (tasked with fighting hackers and viruses). As early as 2013, the FBI, NSA, and CIA continued to use their own networks and systems independently of one another, but by 2024, all of them have been replaced and integrated into the unified Gotham network.

Additionally, Palantir took over the Pentagon contract on the Project Maven in 2019 after Google decided not to continue developing AI-powered drones for use in bombing and intelligence operations.

Donald Trump, during both of his presidential terms, provided comprehensive support to Palantir, especially in the areas of criminal profiling, intelligence operations, and the fight against illegal migration. The Armed Forces of Ukraine use Gotham for reconnaissance and coordination of artillery and missile strikes against the forces of the Russian Armed Forces, and Israel uses it in a similar way against Palestine, which has sparked numerous protests.

Since 2021, Palantir has been cooperating with IBM and their Watson project—one of the oldest and most powerful expert systems. In addition, IBM provides them with computing infrastructure through IBM Cloud Pak for Data. For the same purpose, they also cooperate with Amazon AWS and Microsoft Azure cloud services. Naturally, there is also close cooperation with Cambridge Analytica and Facebook.

IN THE “FIGHT” AGAINST COVID-19

Palantir Foundry is a software platform intended for use in the commercial and civil public sector, but the examples of its application continue to draw attention. For instance, it was used in the National Covid Cohort Collaborative—a secure enclave of electronic health records covering the entire territory of the United States. Foundry was also used by NHS England in the fight (or “fight”?) against COVID-19 in England and for the analysis of the vaccination program.

In June 2021, Foxglove, a non-governmental organization that files lawsuits against tech giants, launched a campaign against Palantir Foundry, claiming that their experience mostly came down to contracts in which people were harmed, not healed. In 2022, Foundry was also used to manage the UK Homes for Ukraine program, which gave Palantir access to data on social workers and housing units in the UK.

In November 2023, NHS England signed a seven-year contract with Palantir to create a unified platform for medical data, sparking strong criticism even from the British Medical Association, the Doctors’ Association UK, and cybersecurity experts. In 2024, protests were held in front of the NHS England headquarters by healthcare workers demanding the contract be terminated.

TESTING DURING THE WAR IN UKRAINE

The third part of this entire ecosystem is Palantir Apollo—a support system for continuous delivery that manages the deployment of Palantir Gotham and Foundry. It organizes the updating of configurations and software on the Foundry and Gotham platforms using a microservice architecture.

This product led Palantir to shift to a software-as-a-service (SaaS) business model and move away from the model of developing custom solutions for individual clients, significantly improving their customer base, making installations easier, and generating substantial revenue. Naturally, Palantir also works on a large number of smaller, local applications, such as the system introduced in 2014—Insights—which extracts customer spending information and demographic data from merchant credit card records.

In April 2023, the company launched its Artificial Intelligence Platform (AIP), which integrates large language models for use with Palantir products. Testing of the platform is being carried out, among other things, during the war in Ukraine. AIP allows users to create LLM models, called “agents,” through a simple graphical interface. These agents can interact with the ontology created by Palantir. This radically simplifies the work of an unqualified operator with the system, as the interaction is conducted in natural language.

THE MACHINE CONTINUES THE WAR

There are also concepts for expanding Palantir’s military segment, as in theory, it enables war to be conducted like a real-time strategy (RTS) computer game. The computer receives data on all enemy movements, predicts their actions, and awaits a command from the operator, who can then issue an order like: “send drones to this and that sector and suppress the offensive,” after which the machine continues the war.

To make deployment even easier, Palantir offers a prototype of a mobile cyberwarfare center called TITAN (Tactical Intelligence Targeting Access Node), developed in cooperation with companies like Anduril Industries, Northrop Grumman, and other contractors. So far, a contract has been signed for the delivery of 10 units to the U.S. military.

TITAN is a truck containing the entire suite needed to operate Gotham—in essence, a mobile data center and command post capable of conducting cyberwar using electronic jamming, satellite intelligence, and autonomous drones. Palantir also owns its own network of intelligence satellites called MetaConstellation, with Skykit terminals, which are used by the U.S. military (especially under USNORTHCOM), and in 2023, Ukraine also gained access to these resources. Palantir offers Skykit Backpack and Skykit Maritime for transporting equipment by individual users or boats. The kit includes batteries, a rugged laptop with branded software, and a quadcopter with computer vision-supported apps.

PALANTIR METROPOLIS: HUNTER OF DISLOYAL EMPLOYE

The latest product is Palantir Metropolis (named after another city from the DC universe, the home of Superman). Metropolis is a software tool for data integration, information management, and quantitative analytics. It is mainly used for internal control within corporations (a kind of mini-Gotham for internal use). Metropolis connects with commercial, private, and public data sets to detect trends, connections, and anomalies. For example, in 2009, JPMorgan used this system to monitor all employees and identify disloyalty to the corporation. Those suspected or predicted by Metropolis to pose a risk to the bank were placed under security surveillance. Later, Palantir Metropolis was merged with Palantir Foundry.

One of the clients of Palantir Foundry is the pharmaceutical giant Merck & Co, which was once the world’s largest producer of codeine, cocaine, and morphine. This company has been involved in numerous scandals, especially in relation to monopolizing medications (it owns hundreds of thousands of patents) and excessively high prices—not to mention the fact that its drugs regularly cause fatalities. For instance, Vioxx, a drug intended for arthritis treatment, killed 2,500 people between 1999 and 2004 due to undisclosed heart-related side effects, while Fosamax, used to treat osteoporosis, caused 4,000 cases of tissue necrosis.

CONNECTION TO THE WUHAN BIOLAB

The company also became known for numerous environmental crimes and even for secretly publishing an entirely fake medical journal in which allegedly independent reviews periodically praised their deadly drugs. In leaked Merck emails published by hackers, a list of doctors who criticized Vioxx was found—those who “needed to be neutralized” or “discredited.” Reportedly, the emails said: “Maybe we need to find out where they live and destroy them.” It’s not hard to assume that Palantir is more than capable of helping with that.

Incidentally, during World War II, Merck & Co participated in developing antibiotics for the U.S. Army and collaborated with the government under the Biological Warfare Program. Merck was one of the pharmaceutical giants involved in the production of vaccines, antidotes, and biosecurity products. In partnership with Black & Veatch, it also participated in the construction of the infamous Fort Detrick and numerous biolaboratories—including the notorious Wuhan Institute of Virology. Given the fact that Palantir Foundry was used for COVID-19 analytics, this connection is more than obvious.

Among other Palantir clients are the Centers for Disease Control and Prevention (CDC), the U.S. National Institutes of Health, and the World Food Programme of the UN. In October 2020, Palantir began assisting the U.S. federal government in building a system to track the spread and deployment of COVID-19 vaccines across the country.