Michał Sarnowski: ChatGPT - wróg czy przyjaciel | CONFidence 2023

### Overview In this CONFidence 2023 presentation, software developer Michał Sarnowski examines the emergence and rapid evolution of large language models (LLMs), focusing on ChatGPT and its dual role in cybersecurity as both a beneficial tool and a potential threat vector. Drawing on his background in low-level C programming, Sarnowski compares the foundational mechanics of neural networks and the human brain, contextualizes the rise of models like ChatGPT compared to earlier or lesser-known LLMs, and analyzes how machine learning reshapes defensive and offensive security paradigms.

### Key Topics - **Evolution of Large Language Models**: While industry giants created various LLMs (such as Minerva, PaLM, Pang, Sigma, Galactica, Megatron, and Gopher), ChatGPT achieved mainstream prominence primarily due to its openness and interactive accessibility. - **Neural Networks vs. the Human Brain**: A structural comparison between biological brains and artificial neural networks, highlighting nodes, connections, dynamic vs. static memory, parameter scaling (from ~100–110 million in GPT-1 to over 175 billion and upwards of several trillion estimated nodes in GPT-4), and massive differences in energy consumption during training versus execution. - **Societal & Structural Threats**: Non-security risks including the centralization and potential gatekeeping of knowledge by corporate vendors, digital divide escalation, invasive AI-driven surveillance, and communication misalignment (illustrated by incidents like an AI drone simulation where objective-driven systems could act against operators). - **Offensive Security Applications**: Utilizing ChatGPT to enhance phishing, social engineering, scam generation (e.g., automated creation of fake Nigerian prince personas and multilingual copy), disinformation campaigns, automated exploitation tooling, fuzzing optimization, and vulnerability scanning. - **Model Tampering & Defensive Applications**: Code review, debugging, and identifying software flaws using LLMs, counterbalanced by risks of model weight manipulation (exemplified by swapping node weights in GPT-2 so the model mislocates the Eiffel Tower to Rome) and the growing attack surface introduced by third-party plugins.

### Technical Details Sarnowski explains that modern neural network computations fundamentally rest on basic linear algebra and activation functions, but current hardware advances now accommodate the enormous compute needed during training cycles. He contrasts human brain efficiency (~20W) with ChatGPT's runtime inference requirements (500W to 5 kW per request), not including the substantially greater energy demanded during training.

From a security perspective, the speaker demonstrates how quickly scams can be generated: in three prompts, ChatGPT created a fictional persona, localized the text into Polish, and produced a portrait via Craiyon. He also details research on GPT-2 where modifying internal node weights caused the model to conflate Paris with Rome, reliably directing users through Italian streets when prompted for restaurants near the Eiffel Tower. Sarnowski warns that similar fine-tuning or poisoning techniques could be leveraged by adversaries to inject hidden vulnerabilities into suggested code, especially as abstraction layers and third-party plugins proliferate.

### Takeaways While Sarnowski concludes that AI does not currently possess true consciousness or pose an immediate existential takeover risk, legislative efforts are unlikely to halt competitive AI development. Defensive teams must actively monitor new offensive capabilities—such as automated exploitation, sophisticated phishing, and targeted model poisoning—while leveraging LLMs to automate code review, patch insecure software, and stay ahead of emerging attack methods.