Police AI platform uses ‘Easter eggs’ to verify human review

As artificial intelligence-powered tools become increasingly common across law enforcement, experts have warned that they need built-in safeguards to ensure that AI-generated content is reviewed by a human before it’s used in the criminal justice process. One AI company, Code Four, has taken an unusual approach: inserting hidden “Easter eggs” into AI-generated draft documents to confirm officers have actually read them.

Code Four was co-founded by George Cheng and Dylan Nguyen, who both dropped out of the Massachusetts Institute of Technology to run the company last year. With seed funding secured through the startup accelerator Y Combinator — and ongoing support through a startup fellowship program run by the software company Palantir Technologies — the pair have added more capabilities, and their company counts nearly 140 law enforcement agencies as customers.

Cheng, who serves as Code Four’s chief executive, said that while doing research at MIT, the company was spun out of a side project with Nguyen, building tools for law enforcement. He described the early phase as a “grind” — they “didn’t know what was happening” at first. To help gain some direction, the pair relied heavily on early law enforcement customers who were willing to let them test the tool in real investigative environments.

And as the two continued working with law enforcement, the need for tools such as Code Four became more apparent, he said.

“Detectives right now, they’re drowning in thousands and thousands of video, audio files and PDF data from phone extractions,” Cheng said. “What we aim for Code Four to do is to narrow that data down to the most important, … clearing the noise from all these different files. So let’s say, for example, there’s 15,000 photos on the suspect’s phone. Our AI aims to essentially be able to locate the 100 most important photos based on that prompt or that query from a detective.”

Along with providing enhanced searches of case files, digital evidence, body-cam footage and reports, Code Four’s software can draft warrants, affidavits, narratives and prosecutor jackets. More capabilities are on the horizon, Cheng added.

“We aim for our platform to be like full stack. So what that means is our aim is that the detective doesn’t have to leave our platform to do anything else, and that they could do everything on Code Four,” he said. “So after they parse through all that information, they find all that information they need. They do all the follow-ups necessary to plug those interviews into Code Four, and they could draft a finalized case report or case prosecutor jacket over in the platform itself.”

The company’s goal of providing its customers a comprehensive toolkit is reflected in its name, which in police and emergency radio communication lingo means that the situation is under control, the area is safe or that no further assistance is needed.

As Code Four has expanded the number of tasks officers can complete, so too have questions grown about the need for human oversight, which Cheng said his company has tried to address through a series of built-in safeguards. One layer of these safeguards is what he calls “Easter eggs” — the hidden details, inside jokes or references intentionally left in films, books and video games for fans to discover. Though the phrase gained widespread popularity after employees at Atari used it to describe a secret message hidden in a popular 1980 game, the term contemporarily refers to any hidden clue or reference snuck in everything from Pixar movies to Taylor Swift’s music.

In Code Four’s platform, Easter eggs take the form of obscure words and phrases that are inserted into generated content. Cheng said law enforcement officers must remove them manually before they can submit an affidavit or case file for finalization: “That basically prevents them from just copy and pasting everything from the AI at face value, and they actually have to make some edits — meaningful edits — to introduce their own words and their own thoughts and emotions.”

“An Easter egg would just be something like a word, for example, ‘axolotl,’ right?” Cheng said. “Unless you’re talking about animals, axolotls are very rare to be like put into a sentence. It could also be like ‘stupendous’ or like like just like Alec Crocodile — it’s just a mumbo jumbo made up word or super hard word that nobody would use in their vocabularies normally. … The random words scatter across the paragraphs, scatter across the reports, and then they have to remove them, and it’s pretty obvious. But we won’t tell them which words there are, so they can’t just like control-F and like search it.”

The company’s second layer of safeguards is hyperlinks, which Cheng said are mapped onto the generated content, almost like citations. Each link can be traced back to a piece of evidence, and the officer has to manually verify the statements and their validity with supporting documents, and remove the links.

Code Four also builds its own AI models. A number of other AI tools share the same generative AI back-end, Cheng said, such as off-the-shelf models or general-purpose tools like ChatGPT. But he said for processing information about criminal cases that can sometimes be gruesome or sensitive — including data related to homicides, suicides, sexual assaults or crimes involving children — consumer-facing AI systems often block or mishandle the content.

And once evidence goes into commercial chatbots, Cheng said, agencies “don’t know what [those models are] trying to do with that type of data.” He said Code Four does not train its model on real law enforcement agency data, and that all processed case data stays in house. That gives agencies the additional ability to audit how the system handles evidence.

Though AI in law enforcement has come under scrutiny from civil rights experts who claim that the systems are prone to producing biased language, inaccuracies, misinterpretations and lies — especially Axon’s Draft One, which the Electronic Frontier Foundation reported last year lacked oversight and transparency mechanisms — some tech companies have tried to use the technology to fill gaps where traditional law enforcement or investigative methods have failed, such as solving cold cases.

But whether the technology is used to turn body-cam footage into an affidavit-ready narrative or to scan decades-old evidence for new leads, Cheng said thinks the use of AI in law enforcement is all but inevitable.

“Within a couple of years … we’re going to see all law enforcement agencies deploy some type of AI. … Whether you like it or not, your officers are probably using ChatGPT,” he said. “Your detectives are probably using Gemini or Claude to try and organize their case files and make their lives easier. It’s just kind of a given.”

Cheng said yet another distinctive feature of his platform is that it complies with the FBI’s Criminal Justice Information Services Division security standards, which pertain to 13 core areas of digital management for law enforcement systems, including data encryption, cybersecurity and access controls.

“If there’s a tool that can help people, people are going to find that tool and use it. The bad thing about them is that they’re not CJIS compliant, and so that’s why I think departments are going to shift,” he said. “Every single department is going to shift to some sort of AI and adopt it. Whether or not that be Code Four or someone else, I do believe all 18,000 departments in the United States will adopt some sort of AI in the upcoming two to three years.”