#ci cd tools | Explore Tumblr Posts and Blogs

jcmarchi · 3 months

Text

Hallucination Control: Benefits and Risks of Deploying LLMs as Part of Security Processes

New Post has been published on https://thedigitalinsider.com/hallucination-control-benefits-and-risks-of-deploying-llms-as-part-of-security-processes/

Hallucination Control: Benefits and Risks of Deploying LLMs as Part of Security Processes

Large Language Models (LLMs) trained on vast quantities of data can make security operations teams smarter. LLMs provide in-line suggestions and guidance on response, audits, posture management, and more. Most security teams are experimenting with or using LLMs to reduce manual toil in workflows. This can be both for mundane and complex tasks.

For example, an LLM can query an employee via email if they meant to share a document that was proprietary and process the response with a recommendation for a security practitioner. An LLM can also be tasked with translating requests to look for supply chain attacks on open source modules and spinning up agents focused on specific conditions — new contributors to widely used libraries, improper code patterns — with each agent primed for that specific condition.

That said, these powerful AI systems bear significant risks that are unlike other risks facing security teams. Models powering security LLMs can be compromised through prompt injection or data poisoning. Continuous feedback loops and machine learning algorithms without sufficient human guidance can allow bad actors to probe controls and then induce poorly targeted responses. LLMs are prone to hallucinations, even in limited domains. Even the best LLMs make things up when they don’t know the answer.

Security processes and AI policies around LLM use and workflows will become more critical as these systems become more common across cybersecurity operations and research. Making sure those processes are complied with, and are measured and accounted for in governance systems, will prove crucial to ensuring that CISOs can provide sufficient GRC (Governance, Risk and Compliance) coverage to meet new mandates like the Cybersecurity Framework 2.0.

The Huge Promise of LLMs in Cybersecurity

CISOs and their teams constantly struggle to keep up with the rising tide of new cyberattacks. According to Qualys, the number of CVEs reported in 2023 hit a new record of 26,447. That’s up more than 5X from 2013.

This challenge has only become more taxing as the attack surface of the average organization grows larger with each passing year. AppSec teams must secure and monitor many more software applications. Cloud computing, APIs, multi-cloud and virtualization technologies have added additional complexity. With modern CI/CD tooling and processes, application teams can ship more code, faster, and more frequently. Microservices have both splintered monolithic app into numerous APIs and attack surface and also punched many more holes in global firewalls for communication with external services or customer devices.

Advanced LLMs hold tremendous promise to reduce the workload of cybersecurity teams and to improve their capabilities. AI-powered coding tools have widely penetrated software development. Github research found that 92% of developers are using or have used AI tools for code suggestion and completion. Most of these “copilot” tools have some security capabilities. In fact, programmatic disciplines with relatively binary outcomes such as coding (code will either pass or fail unit tests) are well suited for LLMs. Beyond code scanning for software development and in the CI/CD pipeline, AI could be valuable for cybersecurity teams in several other ways:

Enhanced Analysis: LLMs can process massive amounts of security data (logs, alerts, threat intelligence) to identify patterns and correlations invisible to humans. They can do this across languages, around the clock, and across numerous dimensions simultaneously. This opens new opportunities for security teams. LLMs can burn down a stack of alerts in near real-time, flagging the ones that are most likely to be severe. Through reinforcement learning, the analysis should improve over time.

Automation: LLMs can automate security team tasks that normally require conversational back and forth. For example, when a security team receives an IoC and needs to ask the owner of an endpoint if they had in fact signed into a device or if they are located somewhere outside their normal work zones, the LLM can perform these simple operations and then follow up with questions as required and links or instructions. This used to be an interaction that an IT or security team member had to conduct themselves. LLMs can also provide more advanced functionality. For example, a Microsoft Copilot for Security can generate incident analysis reports and translate complex malware code into natural language descriptions.

Continuous Learning and Tuning: Unlike previous machine learning systems for security policies and comprehension, LLMs can learn on the fly by ingesting human ratings of its response and by retuning on newer pools of data that may not be contained in internal log files. In fact, using the same underlying foundational model, cybersecurity LLMs can be tuned for different teams and their needs, workflows, or regional or vertical-specific tasks. This also means that the entire system can instantly be as smart as the model, with changes propagating quickly across all interfaces.

Risk of LLMs for Cybersecurity

As a new technology with a short track record, LLMs have serious risks. Worse, understanding the full extent of those risks is challenging because LLM outputs are not 100% predictable or programmatic. For example, LLMs can “hallucinate” and make up answers or answer questions incorrectly, based on imaginary data. Before adopting LLMs for cybersecurity use cases, one must consider potential risks including:

Prompt Injection: Attackers can craft malicious prompts specifically to produce misleading or harmful outputs. This type of attack can exploit the LLM’s tendency to generate content based on the prompts it receives. In cybersecurity use cases, prompt injection might be most risky as a form of insider attack or attack by an unauthorized user who uses prompts to permanently alter system outputs by skewing model behavior. This could generate inaccurate or invalid outputs for other users of the system.

Data Poisoning: The training data LLMs rely on can be intentionally corrupted, compromising their decision-making. In cybersecurity settings, where organizations are likely using models trained by tool providers, data poisoning might occur during the tuning of the model for the specific customer and use case. The risk here could be an unauthorized user adding bad data — for example, corrupted log files — to subvert the training process. An authorized user could also do this inadvertently. The result would be LLM outputs based on bad data.

Hallucinations: As mentioned previously, LLMs may generate factually incorrect, illogical, or even malicious responses due to misunderstandings of prompts or underlying data flaws. In cybersecurity use cases, hallucinations can result in critical errors that cripple threat intelligence, vulnerability triage and remediation, and more. Because cybersecurity is a mission critical activity, LLMs must be held to a higher standard of managing and preventing hallucinations in these contexts.

As AI systems become more capable, their information security deployments are expanding rapidly. To be clear, many cybersecurity companies have long used pattern matching and machine learning for dynamic filtering. What is new in the generative AI era are interactive LLMs that provide a layer of intelligence atop existing workflows and pools of data, ideally improving the efficiency and enhancing the capabilities of cybersecurity teams. In other words, GenAI can help security engineers do more with less effort and the same resources, yielding better performance and accelerated processes.

0 notes

jcmarchi · 4 months

Text

John Forstrom, Co-Founder & CEO of Zencore – Interview Series

New Post has been published on https://thedigitalinsider.com/john-forstrom-co-founder-ceo-of-zencore-interview-series/

John Forstrom, Co-Founder & CEO of Zencore – Interview Series

Zencore is a premier Google Cloud consulting and engineering partner, empowering organizations to succeed through expert guidance, comprehensive services, and a relentless focus on risk reduction and client success.

John Forstrom is Zencore’s C-Founder and CEO, he is focused on helping companies make the transformation to cloud based services.

An early believer in Cloud, John joined AWS cloud management software company RightScale in 2009. While many were doubting the use of cloud computing beyond startups, this experience provided him with a front row seat to the shadow adoption of AWS and value of IaaS in large organizations.

In 2013 John joined Google Cloud as part of the initial business team working with product and engineering on the strategy for large enterprises and digital natives.

When John is not making all the connections between Zencore’s customers, partners and Google he can be found on the nearest body of water (surfing, fishing, swimming, paddling).

For over 5 years you worked at Google Cloud, what were some of your responsibilities and what were some of the key highlights from this period?

I joined Google Cloud in September of 2013 when the Cloud division was just a small startup inside of Google. I was one of the first external hires for a business development team that worked with product and engineering to acquire the initial large, strategic customers.

It was a pretty unique time at Google Cloud in which a few hundred of us (now the business is 35k+ employees) were working hand in hand to compete against AWS, which at the time had a much more mature offering. We were 100% focused on the customer and acted as trusted advisors to the early adopters. These companies knew Google Cloud didn’t have feature parity with Amazon, but found value in having a seat at the table as Google built their products and prioritized features.

The highlight for me was in 2015 when I secured a contract for one of the first billion dollar revenue Google Cloud customers.

Can you share more about the genesis of Zencore and what motivated you as a former Google insider to start a company focused exclusively on Google Cloud services?

I think what we have created at Zencore is pretty special, but the concept is rather simple. More than half the company is ex-Google and we have lived and breathed the complexity of clients going from zero to having a significant footprint in Google Cloud.

We took that experience from inside the machine and created a company to solve the major challenges that clients face as they start their journey or ramp on Google Cloud. For me personally and many of us at Zencore it’s refreshing to not have any limitations between us and doing the right thing for customers every time. We make fast decisions and get the right people involved. Zencore is designed to be a throwback to those early days of Google Cloud.

Additionally, our experience with the partner ecosystem during our time at Google consisted mainly of partners who didn’t start with Cloud. So many of Google’s partners started with Workspace, AWS or IT services and extended that to a Google Cloud practice. The ecosystem has definitely matured, but the opportunity for us was to create a business focused only on Google Cloud engineering from the beginning. Our premise was a partner organization that does one thing really really well would make the biggest impact for Google and its customers.

Zencore has chosen to specialize solely in Google Cloud from its inception. What unique opportunities and challenges does this specialization present in the rapidly evolving cloud market?

When you align your company to a single vendor, there is inherent risk in that approach. However, the risk is not significant given Google Cloud’s growth, broad data and infrastructure product portfolio and investment in Gen AI. We are still relatively early in the global adoption of public cloud services and we are very comfortable betting on Google as one of the two long term winners.

The upside to having an entire company focusing on one thing is we are all rowing in the same direction all day, every day. The collaboration between our engineers is such a powerful part of our culture and that only comes from everyone working to solve similar challenges with our clients. When you have delivered hundreds of Google Cloud infrastructure, data and Gen AI projects, there’s not a lot that we haven’t seen which is really powerful when you are working on a complex, high risk engagement.

You are right that the market moves very quickly and we feel like that singular focus on Google allows us to stay current and provide the most value to our clients.

You emphasize a customer-centric and opinionated approach in your services. How does this philosophy translate into tangible benefits for your clients, especially when considering the integration of open-source solutions?

Zencore’s clients are buying experience from a trusted advisor. When they start a project that has significant risk, they want to know that we are 100% aligned with their interests and sometimes that includes sharing some hard truths. Many times the recommendations we make are to not use a Google Cloud native product because an open source option is the best solution. I think that scenario is more rare than you would think. Google has done a really good job of building managed products on top of widely adopted open source solutions that have low operational overhead and are integrated well with the rest of the platform.

But in each one of these conversations we lay out the benefits and challenges of all the options based on real life experience. The client benefits from this approach when speed is critical. There are so many decisions to make and when we recommend a Google Cloud native product for example, the client doesn’t need to spend time second guessing the decision or wasting cycles doing an evaluation. They know we bring an independent, experienced lens to every decision we make.

Your innovative support model that bypasses traditional ticketing systems has been praised by many. Could you elaborate on how this model enhances operational efficiency and client satisfaction?

I like to joke that one of the biggest benefits of working with Zencore is that none of us have a professional services background. The reality is that we don’t do things because that’s the way they have always been done. Our reseller support offering is a great example of one area in which we have taken an innovative approach.

Many of our clients are mid-to-large size software companies. They have experienced engineers, want to move fast, but sometimes they get stuck.

The last thing they want to do when they have a consultative question is to open a ticket, get triaged by an inexperienced support rep, escalate and have that process take a day or two. It’s a total waste of their time and they end up not engaging with a partner’s support offering.

So we created a model to fit into how they work today. Every client get’s a dedicated Slack channel. On the backend of that channel is the entire engineering staff at Zencore.

So when you ask us a deeply technical question, in 15-30 minutes you are Slacking with an experienced cloud engineer or architect directly who will help to unblock your challenge. In addition, many of the questions we receive are less Google Cloud related than they are about the technology that the customer is connecting to Google like Terraform or a particular CI/CD product. It’s that intersection of the customer’s stack and Google Cloud that can be the most complex.

Direct access to our engineers is like gold to our clients. Rather than struggle with an issue, search stack overflow and get frustrated, they ping a channel and immediately get help from an engineer who has worked on dozens of complex projects.

Our clients have described it as “the next best thing to having direct Slack access with Google.”

What are the most common pitfalls companies face when migrating to cloud technologies, and how does Zencore help navigate these challenges?

We have thought a lot about this question and last year came up with five of the most common pitfalls to cloud migrations.

Not understanding workload needs and insufficient application assessment. Existing workloads might behave unpredictably in a new cloud environment. This can lead to performance issues and failed application migrations.

Insufficient implementation and strategy development. Improper implementation or strategy development can lead to downtime, cost overruns, and a mismatch between an organization’s goals and the outcomes from its cloud implementation.

Security and compliance considerations. Insufficient security and compliance considerations can lead to breaches and fines, as well as a loss of customer goodwill, revenue, and data.

Lack of cost optimization and poor resource management. Without a proper understanding of billing, costs, and how to maximize the return on cloud resource spending, cloud costs can fail to align with business objectives.

Skill gaps. Skill gaps can lead to a domino effect of problems, including poorly designed architecture, inefficient resource allocation, security vulnerabilities, and, ultimately, project failure.

Zencore prioritizes an outcome-based approach that focuses on quickly getting hands-ons with our clients. We want the strategy and architecture to be well thought out, but you cannot spend your time in endless workshops run by consultants. These five pillars best describe our overall methodology.

A deep understanding of the cloud platform. We know Google Cloud inside and out, including key areas like data cloud, machine learning, AI, and Kubernetes.

Proven methodologies. Our streamlined assessment, planning, and migration processes minimize unplanned downtime and reduce the impact on your staff.

The ability to guide the selection of the right intial cloud project tailored for success. We guide you in selecting and planning cloud projects that are set up for success, especially during early phases like evaluating workload migrations.

Expertise in cloud security. We help minimize risks with our deep knowledge of cloud security, protecting you from data breaches and other costly issues.

Hands on development capabilities. We are outcome oriented, and bring the engineering resources needed to get your solution deployed and running in production

With the cloud technology landscape continuously evolving, what emerging trends do you believe will significantly impact how organizations adopt and utilize Google Cloud in the next few years?

I think we are on a journey here in the constantly evolving cloud space. I’ll describe it in 3 steps, and I believe we’re somewhere in between step 2 and 3.

First, we all experienced the shift from Infrastructure as a Service (IaaS) to Platform as a Service (PaaS). Companies are increasingly favoring PaaS solutions because they simplify the development process, reduce the need for managing underlying infrastructure, and accelerate time-to-market. Google Cloud’s PaaS offerings, such as Cloud Run, allow developers to focus more on coding and less on maintenance, which fosters innovation and efficiency.

Second, the rise of managed services is transforming the way organizations handle their cloud operations. Managed services like Google Kubernetes Engine (GKE), Cloud SQL and BigQuery take the burden of routine management tasks off the shoulders of IT teams. This shift not only improves operational efficiency but also ensures higher levels of reliability and security. By leveraging these managed services, organizations can allocate more resources towards strategic initiatives rather than routine upkeep.

Lastly, the integration of generative AI is set to revolutionize business operations across various industries. Google Cloud’s AI and machine learning services, including the new generative AI models, empower businesses to harness advanced analytics, automate complex processes, and enhance customer experiences. For example, tools like Vertex AI make it easier for companies to develop and deploy sophisticated AI models, driving innovation and creating new value propositions.

This is just the beginning of the age of AI in everyday life for organizations running on Google Cloud and it’s definitely where we see a lot of momentum. To that end we built a set of services at Zencore we call Zen AI to help companies building AI applications or integrating AI into their existing processes.

How has your background at Google influenced your leadership style at Zencore, and what key qualities do you look for when assembling your team of cloud experts?

It’s a great question. When you look at the SRE organization at Google the Individual Contributors (ICs) are the most important part of the organization, not the managers. The ICs are highly paid, well respected and make things work without a lot of oversight. They are truly the special forces inside of Google.

What I learned is that if you hire the right people things actually work very well without a dedicated people management layer at our size. I think that one of the most unique things about Zencore is that there are no individuals whose only job is to manage people. We are an assembly of ICs who are still pretty good at their area of expertise that lead others who may be a little less experienced. Creating a company of leaders instead of a company of managers has become a key component to the culture we have created. You respect your manager because in most cases he or she is more experienced at their job and still performing it at a very high level. It’s a very collaborative approach.

From an engineering perspective, we have very high standards. We review so many resumes and they all look similar with the standard Google Cloud professional certifications listed. We generally don’t care how many certs you have obtained. What matters to us when we are hiring an architect or engineer is significant practical experience with Google Cloud. Your experience with migrations, ML ops, building a Kubernetes Operator or your depth with complex data environments leveraging BigQuery are what’s meaningful to Zencore and its clients.

Could you share a case study where Zencore’s approach significantly improved a client’s business outcomes through cloud adoption?

Although migration work is a key component of our business, it’s the data platform engagements that really stand out when you’re talking about value to the business.

One project that really stands out is a complex engagement that involved working with a company that was made up of a diverse portfolio of software brands. They were struggling with operational inefficiencies and an incomplete view of their business due to data being siloed across their various brands. This led to inconsistent data standards and made it difficult for them to gain actionable insights.

When Zencore came on board, our primary goal was to consolidate these disparate data sources and build a highly scalable data platform on Google Cloud Platform. We tackled this challenge through several key initiatives:

First, we migrated their various databases, including Redshift and SQL Server, to BigQuery. This step unified their data landscape, making it easier and more efficient for them to access and analyze their data.

Next, we focused on enhancing their data ingestion and validation processes. By implementing and automating their data job orchestration and integrating CI/CD pipelines, we ensured that their data ingestion was reliable and timely. This setup also improved the data validation checks, which are crucial for maintaining data integrity.

We also standardized their data modeling using DBT, which is an open source tool that allows you to develop data transformation models in a version controlled, easy to understand manner. . This allowed a standardization of data models across the many disparate brands, which made data analysis and reporting much easier for their teams across their portfolio.

Additionally, we consolidated multiple BI tools into a single Looker environment on GCP. This move streamlined their reporting processes and provided a unified platform for generating insights across all their portfolio companies.

The impact of these efforts was transformative. Our client now has a consolidated data environment, which gives them a comprehensive view of their business operations. This unified data platform has significantly improved their strategic decision-making capabilities and operational efficiency. Furthermore, this transformation enabled them to develop a new strategy to monetize their data, creating a new revenue stream and providing them with a strategic advantage in the market.

Looking ahead, what are your long-term goals for Zencore, and how do you plan to evolve your services to meet the future needs of your clients?

The market moves so fast that I’m not joking when I say six months is long term. I think the biggest opportunity for both Zencore and Google Cloud is with Generative AI. We have moved quickly past the hype phase and are now working on projects with real operational value that will go into production. And the value of Gen AI is so compelling that it’s putting massive pressure on organizations to get their data house in order to leverage the technology. The risk of not engaging and understanding the value of Gen AI is that your competition will use it to leapfrog you in the market.

So Zencore is doing several things to address this opportunity. One is to continue to invest in the right architects and engineers that have experience across a broad set of industries and use cases focused on things like RAG, enterprise search and of course Google products like Vertex AI.

You will also see us take a much more vertical approach, which is something historically we have not done. When you solve a specific challenge for one client in an industry using Gen AI, the reality is that you have done 80% of the work to solve the challenge for a significant number of clients in the industry. This is a unique advantage for us when time to market is critical.

Finally you will see us make a significant investment in our data cloud practice. Zencore will always have a 360 degree approach to Gen AI projects and be ready to focus on the infrastructure, security, data pipelines and ml ops to ensure a successful end to end production solution.

Thank you for the great interview, readers who wish to learn more should visit Zencore.

1 note · View note