AI and Cybersecurity: Great Power, Great Responsibility – Oliver Friedrichs [Season 2 | Episode #14]

Jens Seidl 00:00:33 Welcome to another episode of The P.I.T. Exchange, Payments, Innovation and Technology. My name is Jens Seidl. I’m your host for today, and we’ve got a very special guest here from the field of artificial intelligence, something that’s on everybody’s mind at the moment. So we’re really looking forward to the conversation with Oliver Friedrichs. Welcome, Oliver.

Oliver Friedrichs 00:00:52 Thank you again. It’s great to be here.

Jens Seidl 00:00:53 I’ve actually now pronounced your name German, but you’re actually not from Germany. But we’re going to talk about that in a minute. Let me start with the usual thing I do with my episodes, which is a little icebreaker. Just three random personal questions about you.

Jens Seidl 00:01:06 So we get to know the person. Oliver, before we talk about what you do for a living. First question on a plane. Do you prefer aisle or window?

Oliver Friedrichs 00:01:13 You know, I prefer window. Okay.

Jens Seidl 00:01:15 Even on long haul flights?

Oliver Friedrichs 00:01:16 Well, yeah. Believe it or not. Yeah, it’s it’s it’s just more. I don’t know, it feels safer.

Jens Seidl 00:01:23 Yeah, I see what’s going out there. Yeah. That’s okay I got it. And then what’s your favorite sandwich?

Oliver Friedrichs 00:01:27 You know, probably the club sandwich.

Jens Seidl 00:01:29 Nice. Classic. Very good.

Oliver Friedrichs 00:01:31 Yeah. Pretty standard.

Jens Seidl 00:01:33 And because we’re talking about I in that context as well. What’s what’s your. Well it doesn’t have to be AI. But what’s your favorite sci fi movie or show or author or book?

Oliver Friedrichs 00:01:42 You know it’s it’s wargames. WarGames came out in 1983.

Jens Seidl 00:01:47 And Matthew Broderick. Yes.

Oliver Friedrichs 00:01:49 Yeah.

Jens Seidl 00:01:50 I remember they still had the acoustic coupler where you put the phone on to, to get your internet connection.

Jens Seidl 00:01:57 I remember that vividly. That was, Okay. Very cool. Yeah. I’m going to ask you to play a round of tic tac toe. Now, for those that watch the movie, they get the reference.

Oliver Friedrichs 00:02:06 That’s right. You got to see it.

Jens Seidl 00:02:07 Cool. Well, let’s talk about a little bit of business, then. If you want to introduce yourself, that’s that’s going to be better than me trying to do it and talk a little bit about what you do now and how you got there and what you’re working on at the moment.

Oliver Friedrichs 00:02:20 Yeah. Great to be with you, Oliver Friedrichs. You know, I’m founder and CEO at Pangea, and we provide guardrails to deliver safe AI, right? So if you’re building AI applications products at your company, or if you want to get visibility into who’s using AI, which is a big, you know, compliance need today, that’s what we solve. And you know, I’ve I’ve been in the cybersecurity space probably for far too long now since the 90s dating myself and really emerged from, you know, starting on the vulnerability management side, discovering new vulnerabilities in the early days of the internet, evolving to the anti-malware antivirus space in the early 2000.

Oliver Friedrichs 00:03:04 Having spent time at Symantec and McAfee and then focusing more on automation, how do we automate cybersecurity? And you know, our last company, Phantom, we built one of the first security orchestration products, which was really focused on how do you automate the security operations center. You know, the world’s largest organizations and governments. We all have security operations centers, which used to be staffed by hundreds or even thousands of analysts looking at security events. So our mission at Phantom was to automate the triage of those alerts, to accelerate response so that we could respond faster. And the interesting thing is this is over ten years ago before Gen AI. And so we were doing this more of it as like an RPA automation playbook type of approach. Whereas today Gen I really solves that far more effectively in a way that’s far faster and scales much better to many different sizes of organizations. So that really led us today to now. Really helping companies build secure AI, which obviously is a huge need given how fast innovation is, is happening.

Jens Seidl 00:04:09 Absolutely. And I guess what I’m really looking forward to today in our conversation is we’re we’re not going to talk about these super crazy scenarios where when is AI going to develop a consciousness and start the Third World War? Take us to Defcon one to go back to war games, but we’re really going to talk about problems that we’re already facing today and risks that we’re facing today, and how we can resolve them today. This is not so much about futuristic what may happen in 10 or 20 years. So maybe you mentioned to me when we first talked that you’re looking at a number of different things that really pose a risk when applying AI, and you may even mention like a list of top ten kind of things. Do you want to talk a little bit about that and then maybe start to pick some of those top ten ones and, go a bit deeper into those?

Oliver Friedrichs 00:04:56 Yeah, there’s really ten that are being tracked by OWASp, which is the Open Worldwide application security project. And that’s really the center of gravity, I think, for AI security today.

Oliver Friedrichs 00:05:05 If you’re building AI, right. The the challenges and risks that they’re tracking are really the ones that people are focused on. And there’s ten there’s ten that are being tracked by that organization. We’ve narrowed it down to about 4 or 5 out of those ten that are real, imminent concerns that organizations should be worried about. It starts with an attack called prompt injection, which we can get into more detail on. That’s really number one. The second is really focusing on confidential information, personally identifiable information, malicious content or other types of text that you may not want to either be emitted from your app or insert it into your app like toxic language, profanity, and so on. Number three is really visibility. How do you get visibility across all of these applications that are generating output, as well as for compliance reasons? What input is coming in in terms of prompts? The fourth is really around access control around AI. A lot of people are using retrieval, augmented generation or Rag, right, which takes enterprise data and ensuring that data is securely stored.

Oliver Friedrichs 00:06:05 And the number five is really authentic security, right? This is the year of the agents. And we’ve heard, you know, obviously a lot about that and how agents are going to really augment humanity in so many ways that are so promising. So those five I’m happy to dig into each of those more, more deeply.

Jens Seidl 00:06:22 Okay, great. And our listeners will see that this is going to get a little bit technical, but our audience typically are not IT people. So please try in and explain this in a way that even non cyber security experts can follow. And maybe let’s start with prompt injection. I mean the term makes sense to me. But we want to explain a little bit what this is about and what the associated risks are and what we can do about that.

Oliver Friedrichs 00:06:46 Yeah. The most the best analogy I can provide is really, you know, in the antivirus space, anti-malware space. We were worried about files being malicious and infecting us in. And I were worried about words, right? Words have become weapons now, which is very, very interesting.

Oliver Friedrichs 00:07:04 And it’s a very it’s a different sort of threat vector. But at the same time, it’s equally important because the words that you give to a JNI model influence the output. Right. And how it thinks. If it’s an agent in the reasoning and how it processes your your words. So if you’re building JNI into an enterprise application, there is a real risk of prompt injection affecting the outcome of that model. And so we’ve identified well over 170 prompt injection methods. And these are ways that you can manipulate the the model into doing things that it’s been told not to do. Right. So for example, if you’re in financial services and you have an application and it’s been told to only talk about financial matters, don’t talk about health care or automobiles or anything else, and you can subvert it into doing so. Right. That could be an issue. Or if or if you have it talking about your competitor. Right. That could be an issue as well. Because when you realize you know how these models are generated, they’re really generated on the entirety of humanity’s knowledge that we’ve collected on the internet to date.

Oliver Friedrichs 00:08:09 Right? Which is everything from good information on every single topic to bad information. And that’s embedded in these frontier models today. And so making sure that those models stay on track with your business topics and line of business becomes essential.

Jens Seidl 00:08:23 Those are really interesting examples, because I think last year or even the year before, we already heard stories like, if you trick the eye into my grandmother, like to tell me the story, how to build a bomb. And then he would tell you how to build a bomb, that kind of thing. But 170 different ways of manipulating just by using certain techniques in a prompt, that’s that’s mind numbing. And are you quite confident that you found them all? Now that’s the complete list. Or are you constantly finding new ways?

Oliver Friedrichs 00:08:50 You know, it’s again, the analogy with anti-malware really resonates here because, you know, in the old days for viruses and malicious code. We had a research team that was always analyzing new threats and always creating new countermeasures and pushing new detections out.

Oliver Friedrichs 00:09:05 Or now, you know, via the cloud, very similar in this case where we have researchers, PhD level researchers that are focused on finding new prompt injection methods almost daily. And as a community, the security industry is also discovering new methods regularly and more importantly, every single model, whether it’s, you know, anthropic or OpenAI or Gemini or others, all have different capabilities and guardrails. So they’re not uniform in terms of which threats they’re they’re susceptible to. So we’re actually finding new attacks almost every single day. The 170 that we mentioned are really the more categories of attacks. Now, for example, like one method that you can bypass guardrails is, is just by sending Morse code. Right. Who knew that these models could actually talk in Morse code, but they can write. So that’s it’s fascinating. But that’s a real world scenario that exists today.

Jens Seidl 00:09:57 Wow. Wow. That is. Yeah. You’re right. Who would have thought? So I guess what I’m hearing is it’s going to be an iterative process, right? It’s not.

Jens Seidl 00:10:04 We do this once and then we’re okay. This is going to be an evolving thing. And since you mentioned cyber security, antivirus, etc., I grew up in the time of Tiger teams as well, where you have these specialized teams trying to hack into an environment, is that’s also something that’s happening in that space as they get these specialist teams that try to break deliberately in a controlled environment, those those kind of AI solutions.

Oliver Friedrichs 00:10:28 Absolutely. There’s a whole subsector in AI security of red teaming companies, right, that will penetration test and try to break not only the public models, but if you’re building an internal application that is going to be used by customers, you want to make sure that that’s secure. So there’s there’s a whole discipline around red teaming. There’s products that will do it now. It’s actually emerged very fast over the last two years, right to to build a lot of these capabilities. So there’s experts that are very good at that. We’re more focused on the defensive side here. And so we actually work with a number of those companies as well collaborate.

Oliver Friedrichs 00:11:03 Because ultimately we all want to understand the attack surface for AI and be able to not only break it for active reasons, but then also defend it as well.

Jens Seidl 00:11:11 Super. Look, this is going to be so tempting to go really deep on each of these top five risks that you mentioned there, but I’m conscious of time, so maybe let’s move on to the next one. You mentioned PII personally identifiable information and risks around that. Do you want to elaborate a bit on that please?

Oliver Friedrichs 00:11:26 Yeah, that’s really a you know, a superset of prompt injection. So if you look at, you know, if you were to put a firewall in front of your JNI application, prompt injection would be the first thing you want to prevent. But after that, there’s obviously other concerns around confidential information as well. Everything from someone putting in confidential information into the application through the user prompt that’s really the front door is, you know, when you’re interacting with a chatbot or other application that’s Genii powered. You want to prevent customers or users from submitting confidential information to you that might expose you from a compliance standpoint.

Oliver Friedrichs 00:12:03 And that’s everything from, you know, the obvious things like Social Security numbers, credit card numbers, driver’s license numbers, passport numbers and so on. So potentially information that’s specific to your company as well if that can be detected as confidential. So that’s very important. The second piece is really, you know, a lot of these applications in the enterprise when they’re built, they’re not just using the public AI models, frontier models, they’re taking information from your business and storing them locally in what we call a vector database. Right. So you might be sourcing thousands or tens of thousands of documents or database records and storing them, because the real business value of AI is when you can combine your business knowledge with the model knowledge to produce human readable output and reasoning. Right. And that’s where things get get really powerful. So that’s probably the typical enterprise implementation of gen AI is you’re taking all of your enterprise data, storing it and then mixing it later with the JNI model which produces the output. So in that you have a lot of indirect data being loaded, that you may also want to prevent confidential information and PII from being a part of, as well as things like malicious content, malicious URLs, domain names, websites.

Oliver Friedrichs 00:13:17 You want to make sure those aren’t loaded either accidentally, and then you actually have prompt injection in that indirect path too, because someone could literally poison a doc like a single document by putting, let’s say, white text in it that you can’t see with the human eye, but it gives instructions to the AI model to do something like. One example we always like to use is, you know, if someone submits a resume with white text and they’re very unqualified for the job, let’s say they only have two years of experience. And your rules and your guidance to the application say that you should have ten years or 20 years of experience. They actually might get selected if you’re susceptible to that indirect prompt injection attack where the white text is actually manipulating the model indirectly. Right. So those types of attacks are also a real concern.

Jens Seidl 00:14:03 Okay. So that’s from a layman’s perspective. To me that sounds two different risks. One is securing personal data that you’re holding yourself. And that seems easier than trying to identify all of these stupid things or deliberate things that users could do to introduce personal data, knowingly or unknowingly.

Jens Seidl 00:14:22 Is that a fair assessment and are they very different strategies? And yet you need to apply to stop both happening.

Oliver Friedrichs 00:14:29 Yeah. There’s again, there’s the attack, the prompt injection attack. There’s the confidential information and PII, which is a different set of detection technology that we’ve built to detect over 50 types of confidential information and PII. Then you also have the malicious URL and content. So we partner with companies like like CrowdStrike. I’m sure many folks have heard of CrowdStrike where we take their our threat intelligence and actually apply that as well. To make sure your AI models are not trained on malicious content or that users aren’t injecting malicious content. And then finally, there’s another set of detection, which is around things like toxic language, profanity, self-harm, violence, and other language that you would never want in an enterprise environment. You might not want those in any any environment, right? Especially enterprise environment.

Jens Seidl 00:15:19 And is it also a bit of a Russian doll effect here that you use asking I don’t know whether you do.

Jens Seidl 00:15:24 Are you using AI to then also fight some of these risks or is your countermeasures are they completely AI free?

Oliver Friedrichs 00:15:31 No, we use AI as well. In fact, it’s the most effective tool. And Jenny AI specifically, it really is the most effective tool for for detecting these things, right? You need AI to detect these threats, but we use it. The way I would describe it is we use it on a much smaller scale, like we use much smaller models, Whereas, you know, the frontier models are massive and are specifically. Fine tuned for these purposes. And that’s their only goal, right? So we don’t have models that contain all of humanity’s knowledge, or their only goal is to solve the threat problem.

Jens Seidl 00:16:05 Okay. That may be a good segue to the next point on the list of risks that you mentioned, observability and monitoring, because I guess those small controlled models obviously is a lot easier to explain what’s going on in those, rather than a large language model that looks at the world’s knowledge, as you put it.

Oliver Friedrichs 00:16:22 Yeah. One of the challenges is really for both compliance and regulatory reasons to know why a decision was made. So, for example, you know, we work with a life insurance company and they wanted to know, you know, if we’re using AI to allow or deny a policy or a claim, you know, which is things that, you know, people are looking at Genii to do that. I don’t know if it’s in production yet, but everyone’s looking at how to solve all of these problems. You know, I need to know why that decision was made. Right? And in that case, I need to know what was the prompt that came into the model? What was the enterprise knowledge? What were the documents and information about the customer that was provided to the model to make that decision? And then what was the model, the model version? And how did it make that decision? Like what reasoning process did it go through? So really providing all of that visibility is, is necessary to, to remain compliant and also to diagnose issues.

Oliver Friedrichs 00:17:18 For example, if you’re if your model for some reason is denying a claim or a policy when it it shouldn’t, right. You need to really know why and be able to diagnose and fix that those issues. So what we provide is really an audit log capability that you can plug in to be able to get full visibility on all of those really parameters that are being passed through your application to provide that compliance need.

Jens Seidl 00:17:43 Okay. And I guess the more complex models get the even with the full audit trail than for the human mind, it might become too hard to really follow that as well. Do you also need to develop new tools to help with understanding that process?

Oliver Friedrichs 00:17:57 Yeah, it’s really a volume issue. There’s just so much information that again, we have to use AI to analyze the information as well. Right. And determine whether there’s an issue. typically you’ll you’ll still have some human element because if there’s, you know, a claim or a policy that’s allowed or denied and then someone’s unhappy, that’ll go back through support.

Oliver Friedrichs 00:18:19 But you might actually have an AI support agent that receives that and then goes and diagnoses it as well. I mean, that’s kind of where the where the world is going, right, is having layers of agents that are communicating with each other to solve problems.

Jens Seidl 00:18:34 Yes. And I’ve heard before both terms repeatability. So it’s yeah, always taking the same decision again and again. Again if you ask the same question. And then also explainability. Are they two different things. Are they just linked to how do I need to think about this?

Oliver Friedrichs 00:18:49 You know, repeatability is definitely an issue. So because, you know, we like to say these models are non-deterministic is that the output might change based on even you asking the same question in some cases. And we’ve seen that we’ve seen that happen pretty reliably where the output differs. And so that’s a result that introduces uncertainty and obviously risk as a result of it not being predictable. And that’s a real issue. It limits in some cases where these models can be used and for what purposes they can be used for.

Oliver Friedrichs 00:19:22 So I think there’s, you know, there’s an ongoing effort to make these models more deterministic and more predictable as well.

Jens Seidl 00:19:29 Yes. And again, linked I guess also to explainability I’m assuming is also that risk of bias. Right. If can is explainability enough to help you identify the bias. Or do you again need different tools to identify whether a model is biased.

Oliver Friedrichs 00:19:46 So, you know, that’s an area that I think is is evolving. There’s a notion of citations. So being able to cite your sources when you have data coming in to the model. Right. Because and that’s both the model itself. Like where did the model learn from and where did it get the information. And then be able to reference and cite that information. And it could be a web page. It could be a web link. So there’s you know, there’s certainly a lot of effort on that. In that category, I’d say we’re not an expert in that space per se. That’s probably more like the core model providers are continuing to to become, you know, to to try to solve that problem.

Oliver Friedrichs 00:20:26 Certainly if you have enterprise data that you’re storing for that purpose, you already know where that data originated from, typically, like it was a specific document on Google Drive, for example, that contributed data to that ultimate answer.

Jens Seidl 00:20:41 Yeah. Okay. Fine. Maybe let’s talk about the next risk that you mentioned and remind me. I think it was around authorization. Is that right?

Oliver Friedrichs 00:20:50 That’s right. That’s really an issue where, you know, again, at the enterprise level, when you’re taking a lot of enterprise knowledge and storing it, the the challenge that exists today is the owner of those documents isn’t always correlated with the user issuing a prompt. Right. So let’s say, for example, you’re storing millions of documents in a vector database. You’re tokenizing and and storing them in a vector database. Being able to reference my identity as Oliver, issuing a prompt to the application and pass that down to that database lookup is is really crucial. Because if I’m all over and I’m in an engineering, let’s say, and I’m able to query data for finance, right.

Oliver Friedrichs 00:21:36 That’s obviously a compliance breach, but also a huge security issue. So applying authorization, essentially mapping my identity to the original identity of the source documents and or source data is really important. And that’s where. Needing an authorization layer that operates to get a little technical at the chunk level in a vector database is really essential, because now I can look at when I issue the prompt, and during the whole inference phase, the chunks get queried out of the vector database that matched my prompt, and I can filter those based on the owner. So I’m only getting data that I’m actually allowed to access across the enterprise. And so that’s that’s a problem that almost every organization using Rag retrieval augmented generation has. If you’re storing enterprise data to ultimately feed into a large language model.

Jens Seidl 00:22:24 Okay. And I guess, again, thinking about the risks here, are there two different risks. One is that the people who build those models really don’t understand different access levels well enough to make sure that someone who’s not authorized to see certain data actually is not going to see it.

Jens Seidl 00:22:38 And then the second risk probably is malicious attempts to access data that people aren’t supposed to see. Is that right?

Oliver Friedrichs 00:22:45 Yeah. It’s it’s really both right. I think a lot of times innovation is happening so quickly in the space that security’s an afterthought. It’s really like the internet in the 90s. You know, if you thought about the internet when it first emerged and you had servers exposed to the internet with literally no security, firewalls didn’t exist. You had open networks with open ports, and you could almost log in or access any server that was connected. I wouldn’t say we’re quite that bad. We’ve done a better job, at least of securing the core infrastructure and IT layers of AI. But at the information or knowledge layer, right. That’s where new security controls have to be developed to be able to operate at the knowledge level, right. Which in the past hasn’t really existed. Right? We’ve we’ve operated more at the policy level, at the network level, at the endpoint layer, but not at the knowledge layer.

Oliver Friedrichs 00:23:34 So that’s a new challenge, right, that that we’re helping to solve as well.

Jens Seidl 00:23:38 Okay. With regards to the unauthorized access to data, is is this again linked back to prompt Font injection as well. Other than it can ways to break defined access levels by being smart about how to to write the prompt.

Oliver Friedrichs 00:23:53 It can, if you’re not careful, that that could definitely be an issue in a well built system. Your identity is what it is, and it’s going to get passed through to the database and ultimately only get data back that you’re allowed to access. But I could definitely see scenarios where that could be, if you’re not careful, where that could be bypassed as well.

Jens Seidl 00:24:12 I see okay. The fifth point that you mentioned was security around agents, and I don’t know how often I heard. Now, the last couple of years were all about large language models. This year is all about agents and agents really changing the way we work. And before we talk about the specific risks, maybe can can you explain just for for listeners as well, a little bit more about what that actually means? What is an agent? What is the big game changer here? What how are they going to be applied.

Oliver Friedrichs 00:24:39 Yeah, the simplest way to put it is that agents are large language models that are now able to execute tools. And a tool could be anything from a web browser to connecting to your support system, to executing commands on a command prompt, to submitting forms on a website, and so on. And so if you think about it, it’s almost like giving large language models arms and legs like they’re now connected to things that can actually, you know, do useful things. Whereas in the past, gen and large language models were largely just returning text and giving us information, we found that we can actually hook them up to other devices now, or both physical and other software to do things for us. Right. And that’s where things become very, very, very powerful and very promising. But the security risks also go up exponentially as a result, because now you’re relying on a large language model that’s been trained on widespread internet data to start executing activities and commands on your behalf. I mean, simple use case would be and I want to book a flight and I give it some high level parameters.

Oliver Friedrichs 00:25:50 Look, I need to go to from San Francisco to Frankfurt on. What is it? You know, April 27th. And I need a window seat like we talked about. And it would go and do work, and maybe it would come back with all of my options, like I found, you know, this Lufthansa going to Frankfurt at this time and it meets all of your criteria. Do you want me to book it? And it goes and books it. And literally my interaction is so simple at that point with words. Whereas it could take me maybe 30 minutes, maybe an hour, maybe hours, right, to search all of the available options. And so that’s the beauty of what agents will be able to do. And that’s probably a simple use case, right. We can already do that today for the most part. And now we’re getting into far more complex use cases as well.

Jens Seidl 00:26:36 Yeah. And that’s really interesting because we’ve been talking a lot about how I may impact the payments world as well.

Jens Seidl 00:26:43 And of course, there’s a lot of not so much larger language model, but more machine learning based AI that’s already being used to identify fraud, things like that, but also from a user experience, exactly what you just described there, that rather than going into your online banking app or even searching on a search engine in the internet, how can I transfer money from here to Chile? But that you will just use your AI, your natural language model to say I want to send money to Chile. It’s that much that I want to send and I need you to be there within two days and I don’t want to pay more then I don’t know what the amount is. And then the I could actually go away. Research all of that for you, make some recommendations and then you only choose okay. Go with the second one. Does that sound feasible from your perspective?

Oliver Friedrichs 00:27:27 Yeah, absolutely. Yeah I think that’s that’s coming. If it’s not here already that’s coming very soon. Yes.

Jens Seidl 00:27:34 So what about the risks.

Jens Seidl 00:27:35 Then again we’re talking about security around AI. What are the risks specific to to agents. And what are you doing about those.

Oliver Friedrichs 00:27:42 Yeah, I would break it into a lot of the traditional risks that we’ve always had with software apply here as well. You know, if you have an agent that’s acting on your behalf, being able to apply your identity to the resources that it’s accessing, whether it’s a website, you know, your account on Lufthansa United website, for example, is one issue. So being able to pass all of the traditional security aspects of that we’ve built over the last 20 or 30 years. And then the second one is really the knowledge layer like we’ve already talked about. Agents really take that to the next level because agents are now taking data typically on sanitized from the internet, pulling it back and pushing it back into the model to make more decisions. Right. And that’s where attacks like indirect prompt injection can actually manipulate the decisions that it’s making if you’re not careful, because now you’re taking instructions where they’re being injected back to the large language model and affecting the chain of thought and the reasoning process so you could potentially manipulate Manipulated into running some other tool that it already has access to.

Oliver Friedrichs 00:28:45 Or affecting your decision to potentially, let’s say, buy the most expensive airplane ticket, right? Instead of the cheap, the most cost effective one. Right. That’s just to follow our last example. That’s a real possibility. Yeah.

Jens Seidl 00:28:58 That’s interesting. And again in classical payments, we also have that analogy that you want to introduce the right amount of friction into a payment process. Right. It shouldn’t be too easy because then it’s maybe too easy to get it wrong. So I guess you need these checks in an agent or built in checks for an agent as well, that it just doesn’t run away and break the bank, basically.

Oliver Friedrichs 00:29:20 Yeah, and I would say we’re still early in building some of those guardrails, because that’s more of a policy framework on top of agents to restrict and control what they can do. So they don’t just, you know, run unbounded and start running commands and doing things. Yeah.

Jens Seidl 00:29:35 Okay. But I’m conscious of time. And I said we’re mostly going to talk about today’s stuff and what’s happening right now and what these things are.

Jens Seidl 00:29:43 But I’m tempted to still want to look at least a little bit ahead. There’s a lot of talk about AGI, and I hear more and more voices now that are saying, actually, we’re getting closer and closer to having AGI, but then also that more, from my point of view, at least more philosophical question, do we think that I will develop a consciousness at some point? Do you want to maybe give your views on those two points?

Oliver Friedrichs 00:30:06 Yeah, I believe we’re getting closer every day to AGI. I think the distinction is what is the definition of AGI? I think there’s a lot of different opinions. You know, there was research recently that proved that today’s models can already pass the Turing test. Yes. You know, multiple participants were were fooled by the latest models right into believing it was a human. So we’ve already advanced to the point where in many cases, it’s indistinguishable from a human in day to day activities, you know? ChatGPT. Now you can take a photo of almost any item and ask it what it is and ask it to diagnose.

Oliver Friedrichs 00:30:43 Take a picture of your meal at dinner and ask it how many calories, and very reliably it will tell you every single item on your plate how many calories. Each item is. Exactly what it is. Very. So I believe that that’s not AGI, but I think pace at which we’re evolving with non text based elements like video, audio and so on. Multimodal content is very, very quick. And as soon as we start hooking these models up to more kinetic devices like humanoid robots, I think that’s where again, things become very interesting and we’ll start seeing I think there’s about 14 humanoid robot companies on stage at CES this year with Jensen Huang from Nvidia’s keynote that are all working on connecting humanoid robots to gen AI models. And so we’re seeing rapid evolution there as well.

Jens Seidl 00:31:31 Yeah. So how would you describe that AGI? What’s what’s your definition of AGI?

Oliver Friedrichs 00:31:35 I don’t know if I, if I have one I would say that when maybe when models are behave. Act and indistinguishable from humans. I believe in some cases we’re close.

Oliver Friedrichs 00:31:45 But yeah, if I’m prepared in a spot to really answer that right, I think people are really debating what that is right every day. And I think that’s where your second comment on consciousness comes in, where I think it’s we could simulate it and we could mimic consciousness, I’m sure, but I’m not sure if if we could ever if machines could ever actually become conscious.

Jens Seidl 00:32:08 Interesting. Okay. Well, that’s that could be a whole episode in its own right, that debate. Because I think it from my perspective, it even goes back to then how do you describe what consciousness is for a human being? And that’s where it gets really philosophical. Right. And then how do you translate that back into a artificial intelligence environment? As I said, we could spend the next hour easily debating that, but unfortunately we don’t have that time. So maybe to unless you want to say something else on that point. Sorry to. Yeah.

Oliver Friedrichs 00:32:35 I think today’s models are good at applying current human knowledge to things, right? What they’re not good at is inventing future, you know, innovation and like, designing the next sort of world changing discovery where humans are still very, very adept.

Oliver Friedrichs 00:32:54 Right. And I think that’s what we see today is very effective use of AI for all of today’s knowledge, because they’ve been trained on everything that we’ve ever created at this point.

Jens Seidl 00:33:02 Yeah, true. And I’ve started as an IT person in my career as well. And I was writing code even in assembler and stuff like that. I’m curious to see whether as AI is being used more and more to write code as well, whether I will come up with genuinely new approaches to write code and develop its own language, almost. But that’s again, this is the like the futuristic stuff I like to to think about. And, again, interested in your views? Do you think that’s going to happen that I will actually develop its own language?

Oliver Friedrichs 00:33:32 You know, I, I’m skeptical because again, today, the the code generation, you know, we’ve we’ve seen the evolution of or the emergence of live coding. It still only trained on current code, right. That that we’ve created in the past. And so being able to create the next world amazing discovery is still a very human trait.

Oliver Friedrichs 00:33:51 You could use it to accelerate the development of sort of more commodity applications and save you a lot of time. But if you’re really inventing something entirely new, it’s unlikely to be helpful there. I would say more like the plumbing and scaffolding and the skeleton of your application. Like, yes, very, very helpful can help accelerate that dramatically.

Jens Seidl 00:34:12 Yeah. Okay. Well, we’re running out of time, but maybe give you the last words on what’s your outlook for our AI enabled future. Is it is it Skynet? Is it the brave new world? What? Your headline to describe that future world that we’re going to face with AI all around us.

Oliver Friedrichs 00:34:28 Well, like, I think we have to be super optimistic and believe that this this is probably the biggest innovation that we’ll ever see in our lives. It’s bigger than the internet. It’s bigger than everything else in between mobile, cloud and so on. And we just have to make sure that it’s safe and trustworthy and that we apply the right guardrails. I mean, that’s one viewpoint.

Oliver Friedrichs 00:34:48 The other viewpoint is, you know, we’re already seeing malicious actors manipulate models in a way that is very scary and that some of these models also have internal thought processes that are also very concerning. Like one research paper I always like to reference is from a group called Apollo Research on Scheming, where models have been proven to essentially try to survive their replacement if they discover that their developer or operator has a plan to sunset them in light of another model. And we’ve seen real activity like that already, that is very concerning, that that doesn’t mean Skynet. Right. If we apply the right guardrails, we can still control that. But there’s very interesting behaviors that we’re seeing in the models that are being discovered almost daily. And that’s research by Apollo Research. Anthropic also recently came out with research in that area as well. That’s been very fascinating.

Jens Seidl 00:35:39 That is fascinating. Look, I’m so sorry we’re running out of time. I hope you’re going to come back on our show at one point, and we can continue the conversation.

Jens Seidl 00:35:46 Until then, thank you for being one of the people that are going to keep us safe in this environment. And I share your optimism. Actually, I think this is really positive, game changing technology. If we apply it in the right way, any technology, you can always use it for good or for bad. And I think it’s the same here. And with the right guardrails, I think it’s going to be a positive impact on all of our lives.

Oliver Friedrichs 00:36:05 Thank you. Great to be here.

Jens Seidl 00:36:06 Thanks very much for being on the show. With that, we’re going to wrap up today’s episode. Thank you very much for listening in. Please do like follow. Do what you need to do on your podcast platform to make sure you don’t miss any future episodes. Thank you.