{"pageProps":{"posts":[{"name":"million-dollar-ai-engineering-problem","title":"The million dollar AI engineering problem","ogTitle":"","fullTitle":"The million dollar AI engineering problem","ogImageUrl":"https://indent.com/static/og/indent-blog-million-dollar-ai-eng-problem.png","ogImageTitleHtml":"","ogImagePlusLogo":"","description":"How to not leak your AI model.","noSidebar":false,"heading":"","date":"2024-03-13T18:30:00.000Z","content":"\nThe crown jewels of AI: model weights, biases, and the data that trains them. Regardless of where inference is hosted, what model you're using, or cloud provider you use, these `.json`, `.onnx`, and `.gguf` files are one of a company's most valuable assets.\n\nFor companies developing custom models or fine-tuning existing models, they'll invest millions of dollars in engineering time, compute, and training data collection.\n\nMost models operate on publicly accessible data like Wikipedia, Common Crawl, or something like OpenAI's [internal WebText dataset](https://openai.com/research/better-language-models). But like with ChatGPT, the real value comes from the fine-tuning data and reinforcement learning from human feedback (RLHF) that's used to adapt the model to a specific use case.\n\nUsually this data sits in a shared S3 bucket, accessible to everyone in the company. In a very simple case, it might look something like this:\n\n```\n$ aws s3 ls secret-internal-model-archive/models\nFINE_TUNED-openhermes-2.5-mistral-7b.Q4_K_M.gguf\nllama-2-7b-chat-hf-ggml-model-q4_0.gguf\nadded_tokens.json\n\n$ aws s3 ls secret-internal-model-archive/training-data\ncommoncrawl-CC-MAIN-2023-50/\nscale-export-2024-02-23/\napp-rlhf-latest/\n```\n\n\nWhile Llama is a fully open-source model today, it didn't start that way.\n\nBack when Meta first announced LLaMA, they were intending to restrict full access to limited set of researchers and allow people to request access. “To maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases.”\n\nOne week later, [someone leaked the model on 4chan](https://www.theverge.com/2023/3/8/23629362/meta-ai-language-model-llama-leak-online-misuse) — that's a link to TheVerge, not 4chan.\n\n## How to *not* leak your AI model\n\nAfter initial development, the next step is to control access to the model and training data. Unlike most commercial software, AI models are a lot more valuable and a lot easier to leak.\n\nThe primary goal should be to limit access to only the machines that absolutely need it, using a combination of IAM policies and secure virtual networking.\n\nThe second step is to monitor access to the model and training data. This can be done by logging access to the S3 bucket, and using a tool like AWS CloudTrail to monitor access to the bucket.\n\nWhy is the marketing team downloading the fine-tuning data? Why is the model being accessed from a region where we have no employees or customers? Why is [Leon uploading confidential models](https://www.justice.gov/opa/pr/chinese-national-residing-california-arrested-theft-artificial-intelligence-related-trade) to a personal Google Cloud account?\n\nPeople generally will still need access to the model and training data, so the third step is to require justification for access. This can be as simple as a Slack message to a security team, or as complex as a ticketing system that requires approval from a manager.\n\nDepending on team size, the strictness of these controls will vary:\n\n1. Small teams (under 10 people) should require justification by default, especially if you're spending half your seed round training a model. Companies that deal with sensitive data—like personally identifiable information (PII), health data (PHI), and financial info will enforce stricter controls from day one.\n2. Growing teams (between 10 and 100 people) start to tighten control to the models. Infrastructure and model teams still need instant access, while others need a reason or approval.\n3. Large teams (100+ people) who've invested millions into training models — a model leak is an existential risk. Access control is tied to team membership and strict approvals.\n\n## Fine-tuning data security\n\nMost teams use a pre-trained model like Mistral and fine-tune it on their own data. It's much cheaper and faster to get a model that's good enough for most use cases.\n\nFine-tuning data allows AI models to adapt and evolve based on real user feedback over time, resulting in far better performance including with smaller models. Usually this data is based on task completion, user interaction, or even something as simple as a thumbs up/down button shown to users.\n\nFor these teams, the fine-tuning data is the most valuable asset. It's the secret sauce that makes their model better than the competition.\n\nRLHF is what separates you from competitors who are also using OpenAI, Mistral, and Llama models to build their products.\n\nIf data is oil, RLHF is aluminum — a strategic resource for building the future.\n\n## How to set up temporary access for AI\n\nThe best way to secure your AI secrets is to limit access to only the machines that need it, and to monitor and require justification for access.\n\nYou can build a simple internal tool, ticket-oriented service desk, or use a product like [Indent](https://indent.com) to enforce this workflow.\n\nIndent provides a simple API that you can use to require justification for access to your AI secrets, and to monitor and log data access.\n\nIt's also a lot easier if engineers can request directly from Slack and, if approval is required, route notifications into Slack channels. This is a lot easier than having to log into a ticketing system to request access.\n\n## Talk to Us\n\nWe've thought about the broader problem of implementing strict access controls a lot (previously at [Segment](https://segment.com) and [CoreOS](https://coreos.com)) which is what led us to build Indent. If you need any help deciding on the right security architecture or controls for your team, we're happy to help — you can get a demo or [talk to us](/demo).\n\nWe're also building a set of APIs that you can use to build security into your AI products. For example, you can use our Approval API to require justification for access to your AI secrets, and our Prompt API enable AI models to get clarification from users or developers in production.","hero":{"bg":"#2a3543","text":"#8fb5d5","image":"/static/images/blog/content/blog-hero-million-dollar-ai-problem.png"},"authors":[{"id":"fouad","name":"Fouad Matin","twitter":"fouadmatin","image":"/static/images/blog/avatars/fouad.jpeg","title":"CEO"}],"kind":"blog/post"},{"name":"cloudflare-security-best-practices","title":"Locking Down Cloudflare: Security Best Practices","ogTitle":"","fullTitle":"Cloudflare Security Best Practices","ogImageUrl":"https://indent.com/static/og/indent-blog-cloudflare-best-practices.png","ogImageTitleHtml":"","ogImagePlusLogo":"","description":"Keep your Cloudflare account secure with best practices.","noSidebar":false,"heading":"","date":"2024-02-21T10:30:00.000Z","content":"\nFew systems in a company's tech stack are more sensitive than where your traffic is routed. Cloudflare is a popular choice for managing DNS, DDoS protection, and more. (We use it here at Indent!)\n\nPutting some security best practices in place can help you better manage the risk around your Cloudflare account and more easily become compliant with frameworks like SOC 2, PCI DSS, and HIPAA.\n\nHere's the TLDR of what you should od:\n\n1. Least privilege Cloudflare access\n2. Monitor DNS record changes\n3. Set up Infrastructure as Code (IaC)\n4. Enforce security headers\n5. Hardware 2FA keys\n\n---\n\n## 1. Least privilege Cloudflare access\n\nUsers with permissions to add or edit your DNS records can redirect traffic to malicious sites, intercept sensitive data, or launch phishing attacks by altering DNS settings.\n\nTo prevent bad actors from making these malicious updates you should grant access on a least privilege basis.\n\nBy granting temporary access to Cloudflare roles to complete a specific task, and then revoking that access once the task has been completed, it eliminates unused privileged access, making it much harder to successfully breach your Cloudflare.\n\nYou can manage accounts manually or fully automate approvals and provisioning with Indent’s Cloudflare integration.\n\n[Indent](https://indent.com/) is the request access button for work, getting your team access when they need it and automatically revoking it after they’re done. If you have any questions, feel free to [book some time with our team](https://indent.com/demo#schedule) and we’re happy to answer any questions.\n\n## 2. Monitor DNS record changes\n\nThe settings in your Cloudflare accounts can have a big impact on your attack surface. It's important to monitor for high risk activities, like creation or modification of DNS records, API keys, or security sensitive products like WAF, Bot management, or Zero Trust.\n\nThe easiest way to set up that monitoring is to use [RunReveal](https://runreveal.com/) which can collect your Cloudflare audit logs and detect anomalies and critical configuration changes.\n\n> Cloudflare configuration changes are critical to monitor. Cloudflare operates on their customer's traffic on so many layers that configuration changes can have a massive security impact.\n>\n> — Evan Johnson, Cofounder/CEO at RunReveal, former Head of Security Engineering at Cloudflare\n\n## 3. Set up Infrastructure as Code (IaC)\n\nInfrastructure as Code increases your security while also saving you time. Here are a few of our recommendations:\n\n- **Deploy Cloudflare Workers with \\`**_**wrangler**_**\\`**\n - Using wrangler simplifies Cloudflare Worker deployment by providing a command-line interface for streamlined setup, version control integration, local testing, and automated deployment. \n - This creates a more developer-friendly workflow while automatically helping you adhere to Cloudflare's best practices for security and performance.\n- **Configure your Cloudflare account with Terraform**\n - Using Terraform facilitates automation and reproducibility of Cloudflare configurations across multiple environments.\n - It streamlines management, version control, and collaborative deployment while maintaining consistency in managing Cloudflare.\n- **Use least privilege roles to purge cache or call via API**\n - Rather than granting someone full admin access, you can implement low-risk access to manage cache settings.\n - Least privilege roles allow for easier management, version control, and automation of Cloudflare resources, streamlining deployment processes and promoting scalability with efficient resource provisioning and updates.\n\n## 4. Enforce security headers\n\nCloudflare Page Rules allow straightforward configuration by enabling users to set up specific rules for different URL patterns, effortlessly incorporating security headers like Content Security Policy (CSP), `X-Frame-Options`, or HTTP Strict Transport Security (HSTS).\n\nAlternatively, Cloudflare Workers provide a more dynamic and programmable approach, allowing for advanced customization and fine-grained control over security headers based on specific conditions or application logic. Whether opting for the simplicity of Page Rules or the flexibility of Workers, both methods empower users to bolster their website security by enforcing essential security headers effectively within the Cloudflare ecosystem.\n\nHere's an example of enforcing security headers using Workers:\n\n```js\nconst DEFAULT_SECURITY_HEADERS = {\n 'X-Frame-Options': 'DENY',\n // more headers...\n}\nconst BLOCKED_HEADERS = ['Public-Key-Pins', 'X-Powered-By', 'X-AspNet-Version']\n\nObject.keys(DEFAULT_SECURITY_HEADERS).map((name) => {\n newHeaders.set(name, DEFAULT_SECURITY_HEADERS[name])\n})\n\nBLOCKED_HEADERS.forEach((name) => {\n newHeaders.delete(name)\n})\n\nconst tlsVersion = request.cf.tlsVersion\n\nif (tlsVersion !== \"TLSv1.2\" && tlsVersion !== \"TLSv1.3\") {\n return new Response(\"You need to use TLS version 1.2 or higher.\", {\n status: 400,\n });\n} else {\n return new Response(response.body, {\n status: response.status,\n statusText: response.statusText,\n headers: newHeaders,\n });\n}\n```\n\nExplore on the [Cloudflare Docs](https://developers.cloudflare.com/workers/examples/security-headers/) for more information on how to enforce security headers using Cloudflare Workers.\n\n## 5. Hardware 2FA keys\n\nUsing hardware-based two-factor authentication (2FA) keys in Cloudflare significantly enhances the overall security of privileged accounts. Encouraging employees to utilize hardware tokens, such as YubiKeys, over less secure alternatives like SMS-based 2FA, serves as a critical step in mitigating the risk of unauthorized access.\n\nBy providing employees with YubiKeys and actively discouraging SMS-based authentication, Cloudflare users can elevate their account security to a more robust level.\n\nLimiting access for users without hardware keys to only the essential functions ensures that only those authenticated through a secure hardware token can take privileged action, adding an additional layer of protection to sensitive Cloudflare accounts. \n\nThis approach aligns with best practices in access control and authentication, fortifying the defense against potential security threats.\n\nIf you have any questions about implementing these best practices or have questions about Indent, feel free to [book some time with our team](https://indent.com/demo).\n","hero":{"bg":"black","text":"white","image":"/static/images/blog/content/blog-hero-cloudflare-best-practices.png"},"authors":[{"id":"fouad","name":"Fouad Matin","twitter":"fouadmatin","image":"/static/images/blog/avatars/fouad.jpeg","title":"CEO"}],"kind":"blog/post"},{"name":"tasting-notes-from-vercel-evil-rabbit","title":"Tasting Notes from Vercel's Evil Rabbit","ogTitle":"","fullTitle":"Tasting Notes from Evil Rabbit","ogImageUrl":"https://indent.com/static/og/indent-blog-tasting-notes-evil-rabbit.png","ogImageTitleHtml":"","ogImagePlusLogo":"","description":"Hear from Evil Rabbit, Vercel's founding designer.","noSidebar":true,"heading":"","date":"2024-02-16T20:30:00.000Z","content":"\nEvil Rabbit is the founding designer at Vercel, the company behind Next.js, a popular framework for building web applications with React.\n\nIn this interview, Evil Rabbit shares insights into his design philosophy and the principles that guide his work.\n\n
\n Requirement\n | \n\n Description\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n Indent\n | \n
\n 7.2.1\n | \n\n An access control requirements: • Appropriate access depending on the\n entity's business and access needs. • Access to system components and\n data resources that is based on users' job classification and functions.\n • The least privileges required (for example, user, administrator) to\n perform a job function.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • Granular, role-based access • Able to grant access to employees,\n contractors, and third parties\n | \n
\n 7.2.2\n | \n\n Access is assigned to users, including privileged users, based on: • Job\n classification and function. • Least privileges necessary to perform job\n responsibilities.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • adds users to role-based groups • Auto-revokes access after set\n duration\n | \n
\n 7.2.3\n | \n\n Required privileges are approved by authorized personnel.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • Automatic request routing\n | \n
\n 7.2.4\n | \n\n All user accounts and related access privileges, including\n third-party/vendor accounts, are reviewed as follows: • At least once\n every six months. • To ensure user accounts and access remain\n appropriate based on job function. • Any inappropriate access is\n addressed. • Management acknowledges that access remains appropriate.\n Note: This requirement is a best practice until 31 March 2025, after\n which it will be required and must be fully considered during a PCI DSS\n assessment.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • Real-time approvals with no persistent access\n | \n
\n 7.2.5\n | \n\n All application and system accounts and related access privileges are\n assigned and managed as follows: • Based on the least privileges\n necessary for the operability of the system or application. • Access is\n limited to the systems, applications, or processes that specifically\n require their use. Note: This requirement is a best practice until 31\n March 2025, after which it will be required and must be fully considered\n during a PCI DSS assessment.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • Granular, role-based access\n | \n
\n 7.2.6\n | \n\n All user access to query repositories of stored cardholder data is\n restricted as follows: • Via applications or other programmatic methods,\n with access and allowed actions based on user roles and least\n privileges. • Only the responsible administrator(s) can directly access\n or query repositories of stored CHD.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • Granular, role-based access\n | \n
\n Bonus: 8.2.4\n | \n\n Addition, deletion, and modification of user IDs, authentication\n factors, and other identifier objects are managed as follows: •\n Authorized with the appropriate approval. • Implemented with only the\n privileges specified on the documented approval.\n | \nL\" className=\"\" style={{ width: \"232.66949462890625px\" }}>\n • Granular, role-based access with approvals\n | \n
\n One of the checklists from the Apollo missions\n
\n\n Thanks to Tim Neutkens,{' '}\n Jack Hanford,{' '}\n Lee Robinson,{' '}\n Brody McKee and{' '}\n Gordon Wintrob for reviewing drafts\n of this proposal.\n
\n\n\n","hero":{},"authors":[{"id":"fouad","name":"Fouad Matin","twitter":"fouadmatin","image":"/static/images/blog/avatars/fouad.jpeg","title":"CEO"},{"id":"nataliemarleny","name":"Natalie Marleny","twitter":"nataliemarleny","image":"/static/images/blog/avatars/nataliemarleny.png","title":"Software Engineer"}],"kind":"blog/post"},{"name":"stale-while-revalidate-beyond-http-requests","title":"Stale-while-revalidate (SWR) beyond HTTP requests","ogTitle":"","fullTitle":"","ogImageUrl":"","ogImageTitleHtml":"Stale-while-revalidate