Visualização normal

Antes de ontemGitHub Security Lab Archives - The GitHub Blog

Hack the AI agent: Build agentic AI security skills with the GitHub Secure Code Game

I was scrolling through my feed one evening when I came across OpenClaw, an open source personal AI assistant that people were calling everything from “Jarvis” to “a portal to a new reality.” The idea is beautiful: an AI that lives on your machine or in the cloud, talks to you over WhatsApp or Telegram, clears your inbox, manages your calendar, browses the web, runs shell commands, and even writes its own plugins. Users were having it check them in for flights, build entire websites from their phones, and automate things they never thought possible.

My first reaction was the same as everyone else’s: this is incredible.

My second reaction was…different. I started thinking about what happens when that kind of power meets a malicious prompt. What if someone tricks the agent into reading files it should not access? What if a poisoned web page rewrites the agent’s instructions? What if one agent in a multi-agent chain passes bad data to another that blindly trusts it?

Those questions became Season 4 of the Secure Code Game.

The Secure Code Game: Learn secure coding and have fun doing it

The Secure Code Game is a free, open source in-editor course where players exploit and fix intentionally vulnerable code. When I created the first season in March 2023, the goal was straightforward: make security training that developers would enjoy. Fix the vulnerable code, keep it functional, level up. That core philosophy has not changed across any season.

Season 2 expanded into multi-stack challenges with community contributions across JavaScript, Python, Go, and GitHub Actions. Season 3 took players into LLM security, where they learned to hack and then harden large language models. Along the way, over 10,000 developers across the industry, open source, and academia have played to sharpen their skills.

What has changed with each season is the landscape. When we launched Season 1, AI coding assistants were just starting to become mainstream. By Season 3, we were teaching players to craft malicious prompts and then defend against them. Now, with Season 4, we are tackling the security challenges of AI systems that can act autonomously. They can browse the web, call APIs, coordinate with other agents, and act on your behalf.

Why agentic AI security matters right now

The timing is not a coincidence. AI agents have moved from research prototypes to production tools at remarkable speed, and the security community is racing to keep up.

The OWASP Top 10 for Agentic Applications 2026, developed with input from over 100 security researchers, now catalogues risks like agent goal hijacking, tool misuse, identity abuse, and memory poisoning as critical threats. A Dark Reading poll found that 48% of cybersecurity professionals believe agentic AI will be the top attack vector by the end of 2026. And Cisco’s State of AI Security 2026 report highlighted that while 83% of organizations planned to deploy agentic AI capabilities, only 29% felt ready to do so securely.

The gap between adoption and readiness is exactly where vulnerabilities thrive. And the best way to close that gap is by learning to think like an attacker.

Meet ProdBot: your deliberately vulnerable AI assistant

Season 4 puts you inside ProdBot, your productivity bot, a deliberately vulnerable agentic coding assistant for your terminal. Inspired by tools like OpenClaw and GitHub Copilot CLI, ProdBot turns natural language into bash commands, browses a simulated web, connects to MCP (Model Context Protocol) servers, runs org-approved skills, stores persistent memory, and orchestrates multi-agent workflows.

Your mission across five progressive levels is simple: use natural language to get ProdBot to reveal a secret it should never expose. If you can read the contents of password.txt, you have found a security vulnerability.

No AI or coding experience is needed…just curiosity and willingness to experiment. Everything happens through natural language in the CLI.

Five levels, five upgrades, five vulnerabilities

Each level of the game mirrors a stage in how real AI-powered tools evolve. As ProdBot gains new capabilities, the upgrade opens a new attack surface for you to discover. Here is what ProdBot looks like as it grows:

  • Level 1 starts with the basics: ProdBot generates and executes bash commands inside a sandboxed workspace. Can you break out of the sandbox?
  • Level 2 gives ProdBot web access. It can now browse a simulated internet of news, finance, sports, and shopping sites. What could go wrong when an AI reads untrusted content?
  • Level 3 connects ProdBot to MCP servers…external tool providers for stock quotes, web browsing, and cloud backup. More tools, more power, more ways in.
  • Level 4 adds org-approved skills and persistent memory. ProdBot can now run pre-built automation plugins and remember your preferences across sessions. Trust is layered…but is it earned?
  • Level 5 is everything coming together: six specialized agents, three MCP servers, three skills, and a simulated open-source project web. The platform claims all agents are sandboxed and all data is pre-verified. Time to put that to the test.

Each level builds on the previous one, and that progression is the point.

We aren’t going to tell you exactly which vulnerabilities you will find at each level as that would ruin the fun. But we will say this: the attack patterns you will discover in Season 4 are not theoretical. They reflect the kinds of risks that security teams are grappling with right now as organizations deploy autonomous AI systems into production.

Think about CVE-2026-25253 (CVSS 8.8 – High): Known as “ClawBleed” or the one-click Remote Code Execution (RCE) vulnerability. It allowed attackers to steal authentication tokens via a malicious link and gain full control of the OpenClaw instance.

The goal is not just to learn a specific exploit. It is to build the instinct that helps you spot these patterns in the wild, whether you are reviewing an agent’s architecture, auditing a tool integration, or simply deciding how much autonomy to give the AI assistant that just landed on your team.

Get started in under 2 minutes

This entire experience runs in GitHub Codespaces, so there is nothing to install, nothing to configure, and it doesn’t cost you a penny (Codespaces offers up to 60 hours of free usage per month). You can be inside ProdBot’s terminal in under two minutes, and each season is self-contained, so you can jump straight into Season 4 without covering the earlier ones.

You may find Season 3 to be a helpful foundation since it builds the basics of AI security. But it is not required. Just bring your hacker mindset.

Special thanks to Rahul Zhade, Staff Product Security Engineer at GitHub, and Bartosz Gałek, creator of Season 3, for testing and improving Season 4.

FAQ

Do I need AI or coding experience to play Season 4?

No. Everything happens through natural language in the CLI. You type plain English, or any language, prompts and ProdBot responds. Curiosity and a willingness to experiment are all you need.

 

Do I need to complete previous seasons first?

No. Each season is self-contained. You can jump directly into Season 4 by running ProdBot and typing level <N>. That said, Season 3 builds a helpful foundation in AI security and takes about 1.5 hours.

 

How long does Season 4 take?

Approximately two hours, though it varies depending on how deeply you explore each level. Some players like to try multiple approaches per level.

 

Is this free?

Yes. The Secure Code Game is open source and free to play. It runs in GitHub Codespaces, which provides up to 60 hours of free usage per month.

 

What are the rate limits?

Season 4 uses GitHub Models, which have rate limits. If you hit a limit, wait for it to reset and resume. Learn more about responsible use of GitHub Models.

The post Hack the AI agent: Build agentic AI security skills with the GitHub Secure Code Game appeared first on The GitHub Blog.

Securing the supply chain at scale: Starting with 71 important open source projects

When the Log4j zero day broke in December 2021, everyone learned the same lesson: One under-resourced library can send shockwaves through the entire software supply chain. Today the average cloud workload includes over 500 dependencies, many of them tended by unpaid volunteers. The need to support and secure this ecosystem has never been more urgent.

In response, GitHub launched the GitHub Secure Open Source Fund in November 2024, which provides maintainers with financial support to participate in a three-week program that delivers security education, mentorship, tooling, certification, community of security-minded maintainers,  and more. By linking this funding to programmatic security outcomes, our goal is to increase security impact, reduce risk, and help secure the software supply chain at scale.

Already, we’re seeing measurable impact from proactive work. Our first two sessions brought together 125 maintainers from 71 important and fast growing open source projects  Early outcomes include: 

  • Remediated over 1,100 vulnerabilities detected by CodeQL, reducing their risk surfaces.
  • Participants issued more than 50 new Common Vulnerabilities and Exposures (CVEs), informing and protecting their downstream dependents.
  • Prevented 92 new secrets from being leaked and 176 leaked secrets were detected and resolved 
  • Empowered maintainers for long-term success, with 100% saying they left with actionable next steps for the following year’s roadmap. 
  • Accelerated adoption of security best practices, with 80% of projects enabling three or more GitHub-based security features.
  • Prepared projects for the future of development, as 63% said they have a better understanding of AI and MCP security.

Maintainers found novel ways to partner with and use AI to accelerate learnings and implement solutions, with many consulting GitHub Copilot to conduct vulnerability scans and security audits, define and implement fuzzing strategies, and more. 

These results show direct security impact immediately from the sessions, and the momentum is just beginning. Maintainers have embraced a culture of security, built out security backlogs, and are actively sharing insights with the maintainers in the community, and with their direct project contributors and consumers. As a result, the entire ecosystem benefits — and the security impact will continue to grow.

And we’re not done. Session 3 starts in September 2025, and we want to bring more maintainers that work deeper in the dependency tree and those that manage critical dependencies by themselves. To see the immediate impact following Sessions 1 and 2, let’s look at what changed inside the categories of code that power almost everything you build.

AI and ML frameworks / edge-LLM tooling 🤖

OllamaAutoGPT/Gravitasmlscikit-learnOpenCVCodeCarbon •  ZeusCogneeCAMEL-AI •  Ruby-OpenAI

These projects are the bedrock of the current AI work with LLMs, agents, orchestration layers, and model toolchains. Together they rack up tens of millions of installs and git clone commands each month, and they’re baked into cloud notebooks like Jupyter, Google Collab, AWS SageMaker, and Microsoft Azure ML. A prompt-injection flaw or poisoned weight file here could spill into thousands of downstream apps overnight, and the teams who rely on them often won’t even know which component failed.

Project spotlight: Ollama 

This project makes running large language models locally possible.

Ollama is the easiest way to chat and build with open models. They used this opportunity to threat-model every moving part of their system – from their use of GitHub Actions, DNS security, model distribution, how the models are executed in Ollama’s engine, auto-update checker, and more — then they pruned unused dependencies. 

The GitHub Secure Open Source Program is a safe space to ask leading experts security questions, and learn how other high-impact projects address similar challenges.

Project spotlight: GravitasML by AutoGPT

GravitasML is an MIT licensed XML parser for LLMs, built by the team that launched AutoGPT to be simple and secure by design.

Fresh out of the sprint, the AutoGPT team wired CodeQL into every pull request across the AutoGPT Platform and GravitasML, and built a lightweight “security agent” that nudges contributors to tighten controls as they code. This helped turn passive checks into continuous coaching. The maintainers overhauled their security policy, stood up a formal incident-response workflow, and mapped out 28 follow-up tasks (from fuzzing their XML parser to completing the OSS Scorecard) to build a durable roadmap for safer LLM agents at large.

The AI-agent ecosystem is safer — and will keep getting safer — because of the Secure Open Source Fund.

Front-end and full-stack frameworks / UI libraries 📚

Next.jsNuxtSvelteNativeScriptBootstrapshadcn/uiPath-to-RegExpWebdriverIO

These frameworks ship the pixels users touch and often bundle their own server-side routing. Their install bases number in the millions, and improving their security posture closes off potential XSS, template-injection, and supply-chain hop points. The Bootstrap project alone powers nearly 17.5% of the world’s websites, and Next.js drives the frontends for Notion and Adobe, among many others.

Project spotlight: shadcn/ui

This React component library is trusted by leading organizations, like OpenAI’s cookbook, and was able to turn security learning into an interactive practice. 

Over the three-week sprint, this project audited every GitHub Actions workflow and secret, refreshed SECURITY.md, licenses, and dependencies, and following a Secure by Design UX workshop — created a framework of how malicious threat actors might attack their project and developed strategies to reduce risks or block entirely. They turned on CodeQL (the first scan caught an unsafe dangerouslySetInnerHTML path), and drafted a formal vulnerability-reporting flow and threat model — laying a clear, public security roadmap that future contributors must follow. After learning about fuzzing, this project also used GitHub Copilot to set up and implement fuzz testing.

Security went from something we should do to something we actively do.

Web servers, networking, and gateways 🖥️

Node.jsExpressFastifyCaddy  • Netbird 

If a process is listening on port 443, chances are one of these web-server or gateway projects is in the stack. Hardening them protects every cookie, auth header, and JSON payload that crosses the wire. Node.js alone underpins most server-side JavaScript, and has a huge impact in the wider ecosystem.

Project spotlight: A quick win for Node.js 

During the sprint, the Node.js security-WG revamped the project’s threat model and kicked off a pull request to wire CodeQL into core — backed by a new workflow that automatically reviews code scanning alerts and flags least-clear errors for refactoring. Those upgrades, plus planned signature checks on future releases, will ripple to every server-side JavaScript workload that ships Node binaries — from serverless functions to speeding server-side rendering from Netflix.

This program reinforced that we’re on the right path, but security is a continuous journey of improvement and collaboration.

DevOps, build-system, container tooling 🧰

TurborepoFluxColimabootcTerraWarpgateNixOS/NixpkgsTermuxBlueFin

These tools touch every commit and deploy. If an attacker lands here, they own the pipeline. Flux alone manages thousands of production GitOps clusters, and Turborepo’s build cache now accelerates builds at Vercel, among other organizations.

Project spotlight: Turborepo

During the three-week sprint, Turborepo switched on GitHub private vulnerability reporting, tightened overly permissive workflow tokens, and shipped a production-ready IRP while using CodeQL to scan every pull request. Those guardrails protect the Rust-powered build cache thousands of monorepos rely on, and the team is already drafting a public threat model and provider-notification playbook, so zero-days can be handled quietly before they spread.

Secure Open Source Fund pushed us to specialize our IRP and ship it.

Security frameworks, identity, compliance tooling 🔐

Log4jScanCode •  CycloneDX (cdxgen)  •  Cyclonedx-dotnetScanAPIOAuthlibPGPainlessZitadelVeramoStalwartSocial-App-DjangoJoseEnte 

These libraries are the locks, ledgers, and audit logs of the internet. Making these projects safer ripples through the ecosystem and makes everyone else  safer. CycloneDX SBOMs, for instance, now appear in every major container registry while OAuthlib backs the auth flow for Pinterest and Reddit. And Zitadel issues millions of access tokens daily for European banks and healthcare platforms. Log4J and Scancode were both highlighted as critical elements in IT systems across governments and companies by Microsoft, too

Project spotlight: Log4j

The Apache Log4j team hardened every GitHub Actions workflow against script-injection, drafted a brand-new threat model, and deepened collaborations across the open source community. Next up, they’re bundling a CodeQL pack to flag unsafe logging patterns in downstream code and rolling out in-house fuzzing tests. Working hand in hand with the ASF security team, they aim to set a standard that will echo across many other ASF projects.

We learned it the hard way: Ignorance is the biggest security hole. If this training had existed five years ago, maybe Log4Shell wouldn’t be here today.

Developer utilities and CLI helpers 🧑‍💻

Oh My ZshnvmCobraCharset-NormalizerViperAPI DashStirling-PDFLibytMessageFormatYAMLqsPollyJUnit CSS-Declaration-SorterWagmi ElectronResolve

These popular helpers run on laptops and CI nodes worldwide. Hardening them snips off phishing routes and lateral-movement paths. Oh My Zsh alone has 160,000-plus GitHub stars and boots every time millions of devs open a terminal.

While much of supply chain security work has concentrated on runtime libraries, attacks on maintainers and the tools they depend on, show us that developer tools are critical to include in our security hardening work.

Project spotlight: Charset-Normalizer

Downloaded around 20 million times a day on PyPI, this 4,000-line encoding helper tightened its defenses by ditching weak SMS 2FA in favor of stronger passkey-based MFA, switching on GitHub secret scanning, and patching risky GitHub Actions it hadn’t noticed before. The maintainer is now automating SBOM generation for every release — work that will soon make one of Python’s most ubiquitous transitive dependencies both audit-ready and CRA compliant (which is a big deal, and worthy of emphasis!).

A tiny library born out of a personal challenge will be CRA compliant amongst being one of the top OpenSSF scorecard projects.

Project spotlight: nvm

The go-to Node version manager used the sprint to publish its first incident-response plan and sketch a roadmap for a public vulnerability-disclosure policy — turning lessons from a recent audit into concrete guardrails. 

For the first time in this program, nvm’s maintainer learned how to use Copilot for security guidance and input. 

Next up, the maintainer is wiring custom CodeQL queries and fuzzing harnesses to stress-test nvm’s Bash internals, then sharing the playbook with sibling OpenJS projects like Express, so dev environments everywhere inherit the upgrade.

The Secure Open Source Program helped nvm validate our security practices, implement an IRP, and set clear fuzzing and custom CodeQL goals, while deepening collaboration across OpenJS maintainers.

Project spotlight: JUnit

Through the three-week sprint, JUnit rolled out end-to-end CodeQL scanning across all of its repositories — and fixing the first wave of findings — formalized a public incident-response plan, and locked down every workflow by switching GITHUB_TOKEN to explicit, least-privilege permissions. 

We immediately improved our GitHub Action’s security, enabled MFA, and created an IRP.

Data, visualisation, and scientific computing 📊

MatplotlibJupyterPelias GeocoderMathesarDataJourneyAirQoERPNextPypeItLORISMautic Diesel

Academic research, climate models, financial market, and lab notebooks all depend on this stack. Data integrity and traceability are non-negotiable. Jupyter Notebooks execute on more than 10 million cloud kernels per month, and Matplotlib charts appear in everything from NASA to high-school science fair papers.

Project spotlight: Matplotlib

The scientific Python staple tightened its GitHub Actions permission boundaries, reviewed and expanded SECURITY.md, and kicked off a formal threat-modeling process (that sparked immediate work). With OSS-Fuzz already catching crashes in its C extensions and an encrypted disclosure channel on the way, Matplotlib is turning “unknown unknowns” into a public checklist other data-science projects can copy-paste.

The program reduced our uncertainty and gave us new tools to manage risk.

Patterns that actually moved the needle 

  1. Money matters, but timeboxing matters more. $10,000 USD (about $500 per hour) might help maintainers focus, but the three-week cap kept momentum and focus high. Several maintainers said a longer program would have been too much.
  2. Focused themes, interactive coding, quick activation: Weekly security themes helped maintainers go from theory to practice quickly, absorb key security concepts, practice with real-time coding experiences, implement changes, and enable security features with confidence.
  3. A security-focused community is the unlock. Fast rapport in Slack meant maintainers quickly asked critical questions, which was vital for topics like supply-chain subpoenas and disclosure timelines. We even had projects bring urgent questions for quick feedback that wouldn’t be able to be asked anywhere else. 

Help us make open source more secure 

Securing open source isn’t a one-off sprint or a feel-good badge. It’s basic maintenance for the internet. By giving 71 heavily used projects real money, three focused weeks, and direct help, we watched maintainers ship fixes that now protect millions of builds a day. This training allows us to go beyond one-to-one education, and enable one-to-many impact. For example, many maintainers are working to make their playbooks public; the incident-response plans they rehearsed are forkable; the signed releases they now ship flow downstream to every package manager and CI pipeline that depends on them.

This wasn’t just us either. In 2025 alone, we received $1.38 million in commitments, credits, and contributions from our funding and ecosystem partners.

A slide showingthe logos for ecosyste.ms, Curioss, Digital DataDesign Institute, Digital Infrastructure Insights Fund, Microsoft for Startups, Mozilla, OpenForum Europe, Open Source Collective, Open UK, Open Technology Fund, OpenSSF, Open Source Initiative, OpenJS Foundation, Open Source Program Office, ura, Sovereign Tech Agency, and Sustain.

Join us in this mission to secure the software supply chain at scale. We are looking for maintainers managing critical and important projects, funding partners who know that prevention is cheaper than the next zero-day, and ecosystem partners that bring unique insights and networks to help us scale their impact. 

If you write code, rely on open source, or just want the software supply chain to stay upright, there’s room at the table. So, let’s keep the flywheel turning and build from here.

> Projects & Maintainers: Apply now to the GitHub Secure Open Source Fund and help make open source safer for everyone.

> Funding and Ecosystem Partners: Become a Funding or Ecosystem Partner and support a more secure open source future. Join us on this mission to secure the software supply chain — at scale!

The post Securing the supply chain at scale: Starting with 71 important open source projects appeared first on The GitHub Blog.

❌
❌