Ian Gaudette does what many people increasingly do when they want to explore an intellectual question: he asks a chatbot. But Gaudette, a neuroscientist at Florida Atlantic University in Boca Raton, Florida, prefers to keep his research data offline.

“I don’t want Claude to start telling people about my research,” he says.

Why scientists are turning to offline AI

Cloud-based large language models (LLMs), including those behind Claude from Anthropic, Gemini from Google and ChatGPT from OpenAI, represent the state of the art in artificial intelligence. However, these systems may use user input for training purposes.

For researchers such as Gaudette, the performance of cloud-based AI is not worth the potential risks to data protection, reproducibility and research budgets. Instead, they are turning to a growing collection of tools, including Ollama and LM Studio, that allow researchers to download and run LLMs on their own computers.

These programs can help scientists organize literature libraries and run queries to identify useful insights. Programming and setup problems that once prevented researchers from using local AI have become less significant, but two major barriers remain: the hardware needed to run advanced models can be expensive, and the output still requires careful scrutiny to assess its accuracy.

Data privacy is a major advantage of local LLMs

When researchers are asked why they want to run AI models on desktop computers rather than in the cloud, the answer almost always begins with data.

Eric Böhnisch-Volkmann, president of DEVONtechnologies in Coeur d’Alene, Idaho, says he frequently hears this concern from users of the company’s DEVONthink document-management software, which integrates LLMs.

“Medical records, litigation and research are different,” he says. “They just want to keep holding their data and not give it to anyone they don’t trust.”

Those privacy concerns are well-founded. A 2025 analysis found that six AI developers surveyed—Amazon, Anthropic, Google, Meta, Microsoft and OpenAI—trained their models on user input data by default (J. King et al., preprint on arXiv, 2025).

Local AI can support GDPR-compliant research

Ahmed Farooq, a computer scientist at the University of Tampere in Finland, built a local system called TAUCHI-GPT. It allows researchers to search articles, reports and research data protected under the European Union’s General Data Protection Regulation without sending the information outside the university network.

Keeping AI systems within a local or institutional environment can be particularly important when researchers work with medical records, legal documents or datasets containing information about study participants.

Offline models improve reproducibility

Reproducibility is another advantage of running an LLM locally. Cloud-based AI providers can update their models without notice, meaning a query that produces one result today might return a different result next month.

Casey Green, a biomedical informaticist at the University of Colorado Anschutz School of Medicine in Aurora, says a local model can produce more consistent results when it uses the same settings to convert documents into a searchable index.

“It’s important to be able to know that if you want to use it in a scientific workflow,” he says.

Local AI can reduce research costs

Running an LLM locally can also be economical. Jayashree Kalpathy-Cramer, head of ophthalmology AI research at Anschutz, used Qwen, developed by Alibaba Cloud, and NVIDIA Nemotron through Ollama to index 10 years of open-access publications on her computer.

The publication corpus is publicly available and could be sent to remote commercial AI systems. However, because local inference does not incur additional per-query costs, it has changed how she works.

Her system sends department heads a daily briefing that highlights new papers worth reading. The same query is run through three or four AI models, and the resulting summaries are checked against the original sentences before another model compares the outputs.

Research papers in a field are stored in a single searchable index. This allows researchers to cluster and map the literature, as well as ask an LLM where research gaps might exist.

Researchers are already using AI for literature searches

Preliminary findings from a study by Wiley, a scientific publisher in Hoboken, New Jersey, indicate that 68% of approximately 2,400 researchers worldwide use AI for literature searches and reviews (go.nature.com/4rpuvwj).

Some of the earliest research tasks supported by AI included reviewing large collections of studies, monitoring new publications and assisting with writing. Kalpathy-Cramer says these are tasks that local models are already well suited to handle.

Semantic search finds connections that keywords miss

Green built a local database of embeddings—numeric representations of the overall meaning of each document—to process public summaries of grants awarded to universities.

When he received a federal funding proposal focused on HIV research, he searched the database for potential collaborators. A keyword search for “HIV” identified groups studying the virus. However, a semantic search of the same database also found researchers working on antiretroviral therapy who did not mention HIV in their grant titles.

Green then contacted the researcher about a possible collaboration.

Offline AI protects sensitive biomedical data

Dmitrijs Lvovs, a bioinformatician at the Institute for Genome Sciences at the University of Maryland School of Medicine in Baltimore, uses a computer pipeline to analyze spatial data from tumor tissue.

The pipeline produces hundreds of tables containing gene-activity data, information about interactions between cell types and records of treatment responses. To interpret the data, Lvovs and his colleagues previously pasted the tables into Microsoft Copilot, a cloud-based AI service provided by the university.

Copilot could search published literature and suggest what the patterns might mean. But Lvovs wanted to bring that step in-house. He began feeding the tables through Ollama into Google Gemma 4 in a completely offline environment using a tool called llmize.

Because the model is not connected to the internet, Gemma cannot search the literature in the same way as Copilot. Nevertheless, the tables represent people’s data, and Lvovs would rather accept a weaker model than send those datasets to the cloud.

Even when study participants have consented to data collection, he says, “it feels wrong to concentrate all the data in the hands of a few companies.”

The gap between local and cloud AI is narrowing

In the LLM world, “weaker” is a constantly moving target. The performance gap between local and cloud-based models is narrowing rapidly, Lvovs says.

“Currently, the competition is less about the model itself and more about the harness—the actual program that uses the model as a backend,” he says.