Building Local Knowledge Graphs for OSINT

Introduction to Local Knowledge Graphs for OSINT

The talk introduces a practical solution for building queryable offline OSINT (Open-Source Intelligence) repositories using knowledge graph technology and RDF (Resource Description Framework). The presenter, Dr. Pelgrino, has extensive experience in information science, working with organizations like the Pentagon and the US Department of Defense.

Sponsor Acknowledgement

Thanks are extended to the sponsors: Pentester, Osage Industries, Square X, Synagis Labs, Cat Networks, and Red Hut Labs. Ryan Montgomery from Pentester briefly highlighted their free personal security scan service, offering a discount code "hacktheplanet1337" for their paid services.

Approach and Methodology

The presentation focuses on a tools and methods approach, combining traditional AI and knowledge base techniques with modern tools like Large Language Models (LLMs). The core idea is to build local repositories, index them effectively, and query external services without revealing the specific target. This involves collecting more data than immediately needed and managing the larger collection.

Key Challenges Addressed

Rate Limits: External services often impose rate limits, hindering extensive analysis. Building local storage bypasses this.

Query Revealing Focus: Each query to an external service can reveal the investigator's focus. Local caching and data aggregation help obscure this.

Data Integrity: Web content, especially dynamic content generated by JavaScript, can be challenging to scrape accurately. Ensuring data integrity is crucial.

Availability: Reliance on external services means potential downtime. Local caching provides control over availability.

LLM Risks and Considerations

Confidentiality: LLM prompts, like search queries, can be logged by providers, revealing user interests. Running LLMs locally mitigates this.

Integrity: LLMs may struggle with dynamically loaded content or visually complex websites. Ensuring the LLM can process the scraped data is key.

Availability: While local LLMs offer control, running them can be resource-intensive. Choosing purpose-built models can optimize this.

Leveraging Knowledge Graphs and LLMs

The methodology involves collecting sources, extracting information, building a knowledge graph, and then using the knowledge graph with LLMs for analysis. This approach allows for systematic querying and experimentation with LLMs, overcoming limitations of external LLM APIs.

Data Integration: RDF provides a canonical encoding for graph data, enabling multiple analysts to work with a common expressivity.

Relationship Representation: Knowledge graphs, often expressed as triple stores, can represent relationships between entities. Ontologies (like Friend of a Friend) help normalize terms and create schemas.

Querying: SPARQL is the query language for RDF triple stores, similar to SQL for relational databases.

Graph RAG: Graph Retrieval Augmented Generation (RAG) uses knowledge graphs to manage the context window for LLMs, scaling analysis to large collections of source material.

Case Study: Recon Village Website Analysis

The presenter analyzed the Recon Village website, focusing on speakers. Challenges included:

Dynamic Content: Schedule information loaded only upon scrolling or clicking tabs, missed by traditional scraping tools like wget or curl.

LLM Refusal: Some LLMs refused to identify individuals for targeting, demonstrating a potential limitation where corporate policies dictate acceptable use.

Data Extraction: LLMs were used to extract speaker lists, identifying additional speakers missed by manual review.

Knowledge Graph Construction: The extracted data was structured using RDF and an ontology (Friend of a Friend) to represent relationships between people and organizations.

Data Enrichment: Information from Wikipedia (public company data) was joined with speaker data. The age of affiliated companies was added as a key field for analysis.

Workflow and Stack

The proposed workflow involves user decisions on source selection, desired learning outcomes, and ontology selection. LLMs are positioned as first-class analytical participants. The stack includes:

Crawlers: Custom implementations (e.g., using Rust and the rdt crate) to handle dynamic content.

Triple Store: A database for storing RDF data.

LLMs: Local LLMs (e.g., Gwen 3 with VLM) for analysis, requiring a GPU.

Final Takeaways

Bypass Rate Limits: Build local caches to store data, enabling offline analysis and avoiding rate limits.

Curate Local Libraries: Develop a systematic approach to building and curating local resource libraries.

Integrate LLMs: Incorporate LLMs as full participants in analysis workflows for gaining insights over time.

Maintain OBSEC: Address OBSEC challenges with LLMs, ensuring control over their outputs and preventing them from dictating research limitations.

Scientific Repeatability: Employ repeatable methodologies and stochastic evaluation when using LLMs to control their creativity and ensure reliable results.