Annual Audits Are Dead (Long Live Continuous Assurance)

Dan Tangney [00:02]
Okay, David, Stas, you guys ready to dive into this?

David Forman [00:24]
Yeah, let’s do it.

Stas Bojoukha [00:25]
Looking forward to it.

Dan Tangney [00:26]
Yeah, thank you both for making the time. I brought both of you together. I wanted to have a conversation that’s really timely that I’m hearing throughout the GRC community and I know exists within the audit community. And it’s this idea of what happens between audit cycles and how do we get closer to continuous evidence versus just the moment in time point checks. These are motivations that I’m understanding are coming down from the C-suite and the board level no longer feeling safe and secure with that moment in time certification and really wanting a better holistic understanding when it comes to that continuous control monitoring and where do we sit today? Not where do we sit 364 days ago. So without any further ado, why don’t we go through some quick introductions? David, please, the floor is yours.

David Forman [01:16]
Yeah, love to. I love to talk if you have not met me before. So I’ll fill this entire hour here. So please end up interrupting. But David Forman, I’m our CEO and co-founder here at Mastermind Assurance. We’ll go by Mastermind for short, but we are a pure play certification body for ISO standards. Our main kind of hero products are ISO 27001 for information security, 27701 for data privacy and 42001 for obviously responsible use of AI. I’ve been in the ISO space for a better part of 12 years now, worked across three different certification bodies, interacted with about five accreditation bodies that sit on top of that kind of authority. I’ve added the privilege of writing ISO standards, had the privilege of facilitating ISO lead auditor training courses as well. I’m basically as specialized in ISO or a specific subject matter in this confined space as I think somebody can really be. I really enjoy these standards, but also I enjoy learning from others. Whether it be on the platform side, other auditors, auditees, or just people that are interested in getting into kind of breaking into this like GRC space as well. So it’s always good to have those conversations. I’m based in Atlanta and then on a personal note, I have an eight month old at home. His name is Cooper. I’m a big baseball fan. So if you’re familiar with Cooper’s town, Hall of Fame, where it derives from, although my wife might say it was a different meeting, but we are new parents and struggling at best.

Dan Tangney [02:43]
Well, thank you for joining us. Really appreciate it. Yeah, you and I got a chance to meet a few weeks ago down in Atlanta. Really enjoyed meeting you, your team, Andrew. So yeah, thanks for being here. Stas, turn it over to you.

Stas Bojoukha [02:54]
Yeah, that was a great intro. Yeah, let me follow up. Yeah, so I’m Stas. I’m the CEO and founder of Compyl. We are a next gen GRC platform. We’re six years into this. I was a former CISO twice before, but I’ve got everything from CISSP to CISA to CISM to certified ethical hacker. But I also do have the ISO lead auditor accreditation as well. David probably wrote it before you had a chance to actually get your hands on it. But yeah, so I’ve been in this space for a long time as well. I think I’m 25 years now into security, security compliance. Yeah, I mean, we started the company specifically for what this topic is about, just the continuous control monitoring is so important. And now with the use of AI, even when we started six years ago, the possibilities that we have in front of us today did not exist. And there’s some really cool tech going on. And I’m excited to talk about that. On a personal note, I have a three and a half year old and another one on the way. I live in New York City and cycling, beer, hanging out. So yeah, let’s do that.

Dan Tangney [04:08]
You don’t spend much time away from the computer, my friend.

Stas Bojoukha [04:12]
I work all the time.

Dan Tangney [04:15]
Well, hey, let’s start before Mastermind, before Compyl. Both of you were really kind of rock stars kind of forging a path in your own spaces. David, you and I got a chance to speak and you had shared you’ve done quite a number of ISO audits across the gamut. So just kind of David starting here in the ISO space where I know you spent a lot of time. How has this space changed or evolved in the last, let’s just say 10 years as you see it?

David Forman [04:47]
Sure. I kind of gave two perspectives on this one. That’s a little bit more ISO 27000 focus, which I think kind of this group is very familiar with, but also more just say macro view on like why are people even getting certified to any of these ISO standards as well. So if you go back to, I’ll probably say four or five decades now ago, that’s when ISO 9001 for quality management was first kind of coming on the scene. More like a quality standard related to manufacturing than anything IT, but that’s kind of where ISO certification got its start. It wasn’t until I’ll say like 2000s and even 2010s, if you’re talking about the state side that ISO 27001 really started coming to bear. You had some initial CSPs like Amazon Web Services that got certified literally in calendar year 2010, followed by Google Cloud. And then you could see kind of how the rest of the stack started following the CSPs thereafter in terms of hey, what is the benchmark for information security? 27701 data privacy didn’t come out to 2019. That was on the heels of the GDPR go live date, which was May 2018. And then ISO 42001 really, really fresh. That was December 2023 on the heels of that leak of the EU AI Act. If you guys remember that ended up coming out the following quarter. So I’ll say this kind of genre of ISO standards is still fairly new. And that’s what makes it exciting for me. It hasn’t had quite the product maturity life cycle, if you want to call it that, like a PCI has had where it’s been around for 20 plus years at this point, or even FedRAMP, that’s had to go through a couple revisions at this point to kind of refresh itself. ISO is a little bit more of a mainstay at least in my opinion. I’m totally biased. And it goes through regular revisions, but also just how those controls and the requirements within that management system document are written. It’s designed to be flexible for evolutions and technological innovations such as this kind of rise of AI and ML. From my perspective as an auditor 10 years ago when I was doing these audits, it’s literally in clause 4.2 of the standard. It talks about interested parties and 4.3 is around the issues affecting those interested parties. It’s really interesting. We were seeing a lot more, I’ll say proactive efforts to go get ISO certified. It was viewed as, hey, this is a differentiator. I’m going to get this thing and I have something benchmarking me to a direct competitor they don’t have. Now it’s table stakes and kind of getting to this topic of annual audits are dead. I would share the opinion, I guess, that an ISO certificate, I mean, there’s an as-of date on that certificate, it is point in time by nature. I’m not sure how many end readers or at least our clients are viewing that as, hey, this means we have continuous security or continuous monitoring throughout the year. I think if I was to be really honest about the situation, I think it’s viewed more as a tool with procurement to show, hey, we’ve had a third party audit, not necessarily are we continuously monitored. And I’ll say really it comes down to who’s accepting these reports in my opinion. If it’s just a, for lack of a better word, incompetent procurement professional on the other side reading that report and they look at it kind of as a checkbox, that fuels this conversation of like a one time audit that’s here in day 365 of the year and not actually looking at the rest of the picture. Whereas if you have a security professional actually reviewing these reports and they understand what goes into these audits, that’s where you get more of the follow up due diligence questions where an ISO certificate alone, for example, or a SOC 2 examination report alone, for example, would not, I’ll say suffice the full security review for a new vendor. So I’ll pause there in terms of a little bit of kind of the transformation we’ve seen over the last decade.

Dan Tangney [08:29]
Yeah. Yeah, I appreciate that. Thanks for kind of going through that. Stas, putting on your CISO hat, take off your founder hat. When a vendor’s kind of certification actually hit your desk when you were a CISO, what did that tell you about that vendor? What were you looking for? And most importantly, what didn’t it tell you?

Stas Bojoukha [08:48]
Yeah. Also just I’m going to touch on what David said a minute ago, right? Even like 10 years ago, this whole space was completely different. Where the shift really happened for me is like ISO, SOC 2, and GRC for that matter, were really reserved for more enterprise customers. And then everything kind of changed when AWS really made it onto the market and Azure and GCP. And it kind of went from, I remember answering due diligence questionnaires where we don’t have physical data centers, we’re all in AWS, and the customer is losing their minds. Like how do you not have your own data centers? And now if you tell somebody that you have your own data centers, they kind of scratch their heads and they’re like, why? And also even 10 years ago, there was a much smaller pool of auditors that were doing this, right? I mean, it was essentially the big four and then a couple of boutique players, right? And now there’s so many different auditors that are out there and available. And when we were looking at it before, being a security practitioner, I know what goes into those reports. I know how difficult they can be, and especially if you know who the auditor is and what they’re looking for and what the company does. I think that level of, that standard, if you will, of audit really, really changed over the past 10 years as well. There’s now so many smaller auditing firms that have come out of the woodwork that might not necessarily be doing the best jobs. So I think it’s even more critical now to look at who the auditor is, what their accreditations are, how long they’ve been in business. And as David was saying, really follow up with them and see what kind of due diligence they have done or follow up with the customer and really dig deep. So from my perspective, the space has changed a lot. And I think we need to be putting a lot more due diligence into the reports and certifications that are being issued.

Dan Tangney [10:52]
Yeah. Interesting. David, take us through where do SOC 2 Type 1, Type 2 and ISO differ?

David Forman [10:59]
Gosh, you’re really hitting the pain point.

Dan Tangney [11:01]
That’s it, baby.

David Forman [11:02]
I love it. First of all, SOC reporting is an accounting standard. I’m kidding. But when we think about the areas where they are similar first, I think that’s probably a better starting point. I mentioned already if you get an ISO certificate of any of these management system standards, so the special group that you can get certified to as a business, 27001, 27701, 42001, 9001 we talked about earlier, you can get a certificate deliverable or an award from a certification body should you pass what’s called a conformity audit. That certificate will have some dates on it on the bottom of its first page: an original registration date, so how long have you been certified, the most recent version or issuance date, as well as an expiration or expiry date. That issuance date is kind of akin to a Type 1, if you want to think of it that way for SOC 2, where it’s an as-of date. We are providing a conformity decision as a certification body that this scope meets the management system requirements of the standard that you use as your audit criteria. So in that respect, ISO is very akin to a Type 1. Now where I’ll argue: is it truly a point in time assessment? No. Similar to any of you that have a SOC 2 Type 2 examination throughout a given year, typically on a 12 month cycle if you’ve been doing it at any regular frequency, that auditor is not coming and looking for evidence of every single control that you’ve developed against the criteria to be performing 365 days a year. There are some controls that only have four occurrences or even one occurrence throughout that year, like a tabletop test or a user access review. You’re not going to be doing a tabletop test every day across 365 days. You’re not going to be doing a user access review for every system every day either. So some of these controls don’t operate or have an activity throughout the year every single day. And that’s how I measure ISO a little bit too. When we think about how we are going to test ISO 27001 for a given control, we are looking at a look back period up to a year in terms of evidence that we can utilize to gain that reasonable assurance that that control or that requirement has been met. But at the same time, where ISO kind of gets a little bit of criticism: you could have implemented that control yesterday and we audited it today. Again, point in time, is it working right now? But we may do a look back period of up to 12 months to gain that reasonable assurance, depending on the environment we’re looking at. Hopefully that’s kind of a roundabout answer to compare it to these Type 1, Type 2 periods.

Dan Tangney [13:51]
No, that’s really helpful because I think it kind of sets the baseline of what some of this conversation I was hoping to pull out with both of you here. Stas, fast forwarding now, founder of Compyl, six year old GRC player in the mid market enterprise world. Buyers used to ask, are you certified? Now they’re asking, show me your posture today. Talk to me from the product side of this, the motivation and what you’re seeing coming to market and what’s valuable for customers to be able to answer that question when they are getting asked: show me your posture today.

Stas Bojoukha [14:29]
Yeah, no, absolutely. That’s a really good question. Obviously you have to produce the reports and the certifications when you’re being asked. And it does go to procurement most of the time, and then kind of disseminated from there. But we’re constantly being asked for follow-ups, especially some of our large customers, in terms of, hey, are you guys actually doing your UARs four times a year? And can we have evidence of it? Are you guys holding board meetings, whatever it might be? So it’s really evolved into what trust centers and security portals that GRC platforms are offering. For companies that are very particular with which vendors, third parties and suppliers they work with, they are very useful and they can maintain that level of assurance that the organization is doing the right thing. Where it gets a little bit messy is, I don’t know about you guys, but I’ve never seen a trust center with any red lights on it. So everything has to be taken with a little bit of a pinch of salt there. But the general idea of continuously monitoring an environment and having a readout for it, I think is a really good one. I’m just conscious of making sure that it stays above board.

Dan Tangney [15:43]
You mean green doesn’t mean good all the time?

Stas Bojoukha [15:47]
No, no. Green means good, red you never see.

David Forman [15:51]
Yeah, yeah. Yellow either, to be frank. And at the same time, I would say, I’ve seen screenshots of some of these trust centers on the back end for the actual customer. It allows them to push updates to the trust center, which kind of defeats the entire purpose of showing real time monitoring of any of these controls. So if there is a yellow or red light available, I haven’t seen it either. I’d love to see that screenshot.

Dan Tangney [16:17]
David, I’m curious if you’re seeing the same pressures on the certification side. Are customers coming to you asking for things that certifications alone aren’t showing or answering?

David Forman [16:26]
Yeah, it’s interesting. So when we do an ISO certification audit, whether it be initial certification or one of the maintenance years, year two or year three, we have seen, probably the last three years, more customers requesting kind of like a public version of their audit report to also supplement their certificate. So if you’re familiar with the differences of, like, SOC 3. SOC 3 is probably more similar to what you actually get with a certificate deliverable for ISO. You get this one-two page document from ISO that’s technically the public report, or public document, that you would attach to a trust center or send to a prospect without an NDA. Now you’re getting into the NDA side of this where a trained reader will look at that certificate and be like, okay, cool. I see the product I’m procuring from you is technically part of this management system scope, but it’s based on this thing called a statement of applicability. Where’s that document? And they’re going to ask follow up questions: I want to see the statement of applicability. I want to see the full audit report that actually shows the findings in there, because with any ISO certificate you could have been blown up as a customer and have 15 non-conformities in there, corrected them in time, and still you got the pass. And there are customers, I’ll say, that are trained on this, typically former auditees themselves, that understand that report has more of that information in there that you would otherwise get in a full SOC 2 report, different from a SOC 3 that’s abbreviated. So to answer your question, I’m seeing more due diligence follow up, which I think leads more to that signal that the certificate alone is not sufficient. And likewise, I’d say our customers are bundling it with other audits too. So 10 years ago we would have mid-market SaaS companies that all they would do is ISO 27001. Now they have an information security third party assurance report for every sector that they operate in, even some diverse geographies they operate in too. ISO 27001 alone is just not being accepted by every type of customer depending on how they’re regulated or the jurisdiction they operate in.

Dan Tangney [18:33]
Yeah, yeah, really good point. Stas, again, going back to the product side of things. I think the fire drills, the weeks leading up to an audit, those have happened for quite some time. I don’t think that’s new. And even the weeks after, you’re kind of having the postmortem conversations around it. But what I think is different in recent years is when prospects are looking for GRC tools, the alleviation on the evidence collection side of things is almost like table stakes. They are believing that if they move forward with a GRC tool that has integrations, then they will have some relief, some reprieve, when it comes to the amount of evidence that has to be collected. The caveat there is what evidence is actually being collected, right? What is the automation pulling? What’s the integration pulling? And I think that’s some of the really nuanced questions that we’re starting to hear, that is a signal of the maturity of an organization and the evolution of where GRC platforms have gotten to, and how they in some ways at times have miseducated the market thinking, hey, plug in your AWS environment, we’ll automate the evidence collection. And then after you go through an audit or two you start being asked by your auditor, hey, where did this evidence come from? And why was this evidence pulled? This isn’t relevant. So it kind of is a challenge when buyers are looking to really have a robust security posture and make sure that evidence is relevant to the controls it’s being pulled for. So could you talk a little bit about that, on that side of relevance and freshness of evidence, how you’ve tackled that and thought about that on the product side to make sure when we pull evidence it’s actually relevant to the control?

Stas Bojoukha [20:30]
Yeah, I’d be happy to. In terms of audits, it’s not everybody’s job to be a part of an audit or provide evidence to an audit. You’re taking away from other people’s time, right? You’re taking away time from engineers, from C-suite stakeholders. And you wanna make sure that when an audit is done it’s as efficient as possible. At least that’s the goal for most companies. What we’re seeing with that efficiency though is a lot of times you plug in these integrations, right? You plug in your Azure, AWS or GCP, everything goes green, evidence starts being collected. And when it’s time for an audit you present the evidence that has been collected. Where the rub comes in is, I’ll give you an example that comes up all the time. GitHub, right? Your repos that you’re looking for, all the change requests, all the pull requests. We wanna show evidence that branch security is in place and two people reviewed it and then it gets pushed in and gets tested, QA, all that. Great, but I have 40 repos, let’s say, and only one of them is production. Yet we’re getting all the data from the 40 repos and then passing it over to the auditors and they’re like, we’re not interested in any of them unless they’re production. And that becomes a real sticking point, because then you’re having to ask somebody to go and redo it by hand, or worse yet, that evidence has only been collected for 90 days anyways and then you can’t actually pull it back. So having the ability to get really narrow in terms of what’s in scope of the audit becomes really valuable as you scale. And other examples of that as well: if you’re doing Jira tickets and not looking at the right project and pulling down the wrong data, or being able to remove service accounts that don’t have passwords set to expire but they’re only read-only accounts used for something very specific. Just that level of detail can save a lot of time and a lot of explanation. So as you scale, that becomes more and more important and really allows you to focus on the things that you do care about. Like you do want to get alerted whenever somebody uses the root account for whatever reason. Why did that happen? Let’s take a look at it. An HR record gets re-enabled for some reason. Why did that happen? Who’s that user? Having that ability to proactively monitor an environment, even if it’s not related to an audit, just being able to monitor it, see things that are happening in real time, understand what’s happening, it just goes a really long way. And this is the type of technology that’s available to us today. It was hard to do this three years ago before AI, but now just being able to see anomalies in real time, that is really, really valuable. It is important to have that flexibility both for the company itself, but also for audit purposes, making sure that you’re auditing the right controls and that you’re providing the right evidence for them as well.

David Forman [23:31]
Dan, if I may jump in here. So I like to give this example to more non-technical people, like my wife.

Stas Bojoukha [23:39]
She might be listening, be careful.

David Forman [23:41]
I guarantee you she is not listening. But I will tell you this. If you use these personal finance apps, like Mint was around for a while, it got deprecated, there’s Wealthfront now, NerdWallet. Essentially you connect your credit cards, your bank accounts there, and then you get a full financial picture of your personal accounts at a given time. Same kind of problem we’re seeing with utilization of some of these platforms by auditees. Using your GitHub repo example, or an Azure tenant. If you think about initial configuration setup of these, a lot of these platform providers have somewhat of an implementation support services team for initial onboarding. And typically that part goes pretty okay for a customer, because that’s kind of a playbook you’d follow to get them initially set up, especially depending on the common integrations your platform has. Where it breaks down is typically in year two. Because 12 months have gone by, there’s new technology that’s been procured by the company. Maybe they haven’t set up an appropriate integration for it. Maybe they did a workaround and it’s not exactly how the platform designed it to be. Or it’s an existing application and they haven’t updated it for current visibility. So we go back to this finance apps analogy. Imagine you have reported your main credit card lost or stolen, and your bank, JPMorgan Chase, sends you a new credit card in the mail. That credit card has a new credit card number and it’s connected through a Plaid-type middleware service to your app. And you have not authorized that app to look at the transactions for that new credit card. All of a sudden you have a very different financial picture if the card that you mainly use for gas and restaurants is not being ingested by this platform. It’s the same thing for these compliance automation platforms as well. Hey, we have this single GitHub repo that is our only source of production branch protection rules that we would care about as an auditor. However, in year two, we brought in a new head of engineering. He blew up the entire stack and now we have 40 of these repos and we’re still only looking at one. That would give you a false image of branch protection rules, just as an example of a control we would audit. And so I’d say once you learn in this space that the entire world is visualization tools built on top of JSON files, you really quickly figure out where some of these dead bodies can be when you’re maintaining a compliance automation platform. That’s probably the biggest risk for a lot of these auditees: they don’t actually understand how the data is being intaken by these applications to create this pretty dashboard that they’re looking at. They only look at the dashboard, and dashboard alone. And so then it becomes an education exercise between both platform providers and auditors to say, hey, we’re gonna get down to the logging detail here to figure out where it’s pulling from, what visibility to which tenants or other instances in these applications, and then we’re gonna audit whether or not the visibility is correct. I’ll tell you, the number of times we get findings around visibility of these tools, it’s kind of bad, because it shows that there’s not this ongoing customer success element happening between most of these auditees, especially in the higher churn businesses, and the actual platform providers too. So that’s a huge risk we see when organizations go all in on the platforms and don’t really actually understand what they’re doing with the platforms. That education component’s difficult.

Stas Bojoukha [27:15]
No, I think that’s a great point. Being able to stay on top of the ecosystem and specifically your tech stack is very difficult, especially when you have shadow computing. And we’re seeing it at such a different scale. What people are really worried about right now is like ChatGPT, and signing up for Anthropic accounts, per person accounts. So the best way that I’ve seen to stay on top of that: generally if the organization has good single sign on controls, being able to monitor where the single sign on controls pop up, pull it down and make sure that that new vendor is now in scope and that you’re putting in proper controls, but also connecting it to the procurement team, because these companies have to get paid, so pull a feed out of that. But even that still takes a lot of work. But I totally hear you. The drift and how quickly businesses are changing is definitely an issue.

Dan Tangney [28:30]
For sure. We’ve had three questions come in. I’m gonna save them till the end and if we’re able to weave in and answer them along the way we will. David, curious from the auditor chair, can you tell when a company runs a program year round versus the overnight fire drill? And if so, what are the tells?

David Forman [28:51]
Oh my gosh. For everyone that’s listening, I was not given these questions in advance. So it’s kind of funny. Can I tell when basically an organization is fire drilling it? I’ll say there is definitely stuff that we can figure that out. I mean, first of all, we see dates on activities like the management review, for example, and you can see when the risk assessment was entered, when the scope was finalized. And then generally speaking, just when the policy review cycle is happening. I mean, that’s kind of a terrible answer, but you can look at version control in documents and figure out it was all done in the last 30 days. So that’s probably the easiest way to figure it out. For more of an auditee sitting on top of a platform, I’ll tell you something that raises questions is when the users on the auditee side are limited to one or two individuals for the company. I don’t care if they’re even 20 people. It really shows that compliance is being managed by one or two persons. And then you start getting into their signing activity. And you see it’s one person signing in every 90 days to basically correct a red light for an overdue user access review. That’s a good sign too. So these platforms can go both ways in terms of helping you with your audit. They can also show your login activity. And that’s always an interesting little trail to go down to figure out who’s actually using this.

Dan Tangney [30:08]
Fair enough. Stas, I don’t want to turn this into a product demo, but I want to talk about theory, continuous control monitoring. Walk us through that. When customers are asking how we are providing continuous control monitoring, and the value of that, what’s actually taking place? Just walk us through that.

Stas Bojoukha [30:33]
Yeah, for sure. The traditional way that we’ve been doing it: we connect in these platforms, we write out our baselines. Simple things like, do they have passwords? Yes they do. Are they 12 characters or more? Yes. Do they have MFA enabled? Yes. Just checking those and continuously monitoring if those ever change on the user, or if somebody disables it, and being able to pull that up. The problem with a lot of this is they’re pull requests, right? Not pull requests in the sense of GitHub, but you’re pulling the data on a set schedule. Sometimes that’s daily, weekly, monthly, quarterly, depending on what that query is looking for. So first, there are chances of missing data capture because a change has occurred and somebody put it back. Most of the time you can see that through last edited dates and things like that, so you can determine what’s changed. That’s the first evolution of where we were with continuous control monitoring. Where we’re going is: there’s so much data being ingested and there are really interesting ways to actually interconnect data sets, right? Just really simple things. The amount of times that I’ve seen organizations have a SIEM that’s not connected into the entire ecosystem, or nobody’s reviewing it, or there are alerts and they’re being missed because there’s too many things for somebody to monitor in one place. The opportunity now to really consolidate your tech stack into a single GRC platform, I think we’re really seeing that. And the benefit of that is you’re also connecting platforms that have never really been connected before. IT would operate in one stack, compliance would operate in one stack, you have legal operating in another stack, and you have your HR in another stack, and very rarely would those systems ever be mixed. And I swear, I have a point. What we’re seeing is this really cool use case came out where we’re working with a large brokerage, and they have a requirement that they have to trade on site. If you’re trading stock, you have to be on site. It’s one of the requirements of the SEC. And we had access to their HR system. We have access to their turnstile system, and then we have access to the trading system. So what we ran is: hey, how many people are trading that are not in the office? Check the HR record for a person who doesn’t have PTO. They haven’t come through the turnstile, but yet they’re trading, or there are trades. Just really interesting use cases like that. They were picking up stuff that we’ve never been able to pick up before, because these systems have never been connected. But then on top of that, instead of having these rules be written out and implemented by hand, you’re just having AI review all the data coming in and putting these conclusions together and saying, hey, this might be worth you actually checking on. Or, hey, we noticed this vendor had a breach, you have a renewal coming up, maybe you shouldn’t renew it. It’s becoming a lot more proactive than reactive, and that’s what I really like about it.

Dan Tangney [33:58]
David, I’m curious on your side, with the auditor lens here. I think it’s a little unfair, I think the GRC community makes it challenging for auditors having to learn all of these tools: how to conduct audits, how to pull evidence out, where the evidence lives, is the evidence up to date, and so on and so forth. You talked about the maturity, and that’s an area where you feel GRC platforms can improve, those year two opportunities when environments are changing. Talk to me about your thoughts outside of just the implementation and ongoing customer success side of things. On the actual tech stack side, what is your hope to see from GRC platforms? Maybe you’ve seen it already, or what have you seen recently that you really like?

David Forman [34:47]
Yeah, I’ll give a biased opinion here. But if you think about ISO standards, one of their primary focus areas is this idea of a management system, which is a governance program. And if you think about platforms, when they first start up, they all kind of go the same route. They start on specific frameworks like SOC 2, and that’s popular in the US at least, if you’re gonna be a North American platform provider. Then you get SOC 2 under your belt and you’re like, all right, we’ll do HIPAA now. HIPAA seems easy, doesn’t get updated that frequently, so that’s pretty stagnant. And then they say, oh, there’s so much overlap here with the Annex A controls, we found ISO 27001, why not that one next? Then they go the PCI route and HITRUST. And then they’re like, okay, are we finally ready to do public sector? We’ll go FedRAMP, guys. So that’s kind of the roadmap you see. It’s very framework specific. And you guys were talking about it earlier today too, platforms become very table stakes when you talk about it from the feature side. You should help the auditee collect evidence, that’s use case number one. Use case number two is you should support the frameworks that they’re working in. And use case number three is obviously around integrations. So whenever you see platforms boasting on LinkedIn about new things they’ve shipped in the last month, it’s always one of those three things: either help with evidence collection, expanded to additional frameworks, or they’ve shipped new integrations or other ways you can interact with maybe their MCP. So going back to the auditor wish list, to your point. What I’d like to see more of is a focus on the governance side. Everything we’ve talked about so far is very heavy on crosswalking frameworks to where they overlap, which in ISO world is typically the controls. It’s not the governance. And it’s a hard conversation to have, because when you start getting auditors or auditees that are trained in multi framework, so outside of ISO, everyone tries to start mirroring controls. Oh, I understand risk and I understand how a control curbs risk or limits risk, whatever you wanna call it. But they miss the point with ISO, where it’s focused on the governance side too. I had a client conversation this morning: we’re already SOC 2 Type 2, we’ve been SOC 2 Type 2 for five years, we never get exceptions. Awesome. We have security, availability, confidentiality. Great. And they said, we should probably be a really easy fast track to ISO 27001, we think SOC 2 is harder anyways. I’m like, you’re kind of correct, and you’re kind of wrong. Where there are overlaps between those two schemes, are they one-to-one overlaps? No. In my opinion, SOC 2 is actually more stringent for similar controls or criteria that mirror up to Annex A of 27001. I’ll give an example. In ISO 27001, there’s this control around secure authentication, basically username password. There’s no requirements around password attributes. There’s no character length limit for production systems, no rotation complexity requirements, no can’t-use-the-last-four-passwords for the same system. Whereas when you get to SOC 2, you could obviously build a control around criteria to be that stringent based on risk, based on the data that’s in that system, whether that system is prod or not. And I think you can get something far more robust than what ISO 27001 black and white would require. So on that topic, you’re right. Where you’re wrong is saying, hey, we have 85% overlap with ISO 27001. I think you have 85% overlap between the controls you created for SOC 2, and they could overlap with parts of these requirements in 27001. However, your risk assessment, if it’s only based on SOC 2, is probably pretty bad, because SOC 2 has got a pretty poor standard for what they consider an information security risk assessment. If you are only depending on SOC 2, you likely don’t have an internal audit function as well. So you’re depending on this, to your point around this entire webinar, this once a year, probably three day audit with an external auditor that barely knows your environment to flag all this stuff to you. In reality, you want that second party audit, because that’s the police that actually are in your environment throughout the year. And they can be a little bit more difficult on you so that they can air out that dirty laundry, hopefully before the external auditor ever comes as well. And now that external auditor’s just checking that internal auditor and saying, hey, is this self policing activity? Is it effective? That is a far better assurance in my opinion than trying to just say, hey, we crosswalked this between these two frameworks, we’re covered. Each of these frameworks were never meant to be mashed together like this. We did this in this industry out of convenience, out of profit. I mean, there are some billionaires that have been made off of these platforms, right? And I’ll say billionaires have been made off the audit side too, to be honest. And so when you have the market pulling in that direction and saying, oh, there’s commonalities between these schemes, these frameworks, I think there’s some truth to that, but I would still hesitate to say one-to-one control, that secure authentication control in 27001 is the same thing as an equivalent criteria in SOC 2. And that’s where I’ll challenge people as an auditor. It’s not a no, it’s an explain it to me based on your risk. Because not every single client is gonna be able to get away with that statement based on how we at least assess that similar control.

Stas Bojoukha [40:36]
So maybe qualifying the statement with up to 85% overlap.

David Forman [40:42]
I love that actually. If someone can get in touch with Chris Halterman of the AICPA, that’d be great.

Dan Tangney [40:48]
Okay, Stas, be careful here, because we have a live auditor on the phone. David has feelings too. But I’m curious, let’s flip this on its head. What would you like to see in the auditor world? Updated, changed, adjusted. I know we’ve seen and felt this race to the bottom on some of the audit side, and that’s caused pressures that have affected different parts of businesses. What would you like to see different or evolved or changed?

Stas Bojoukha [41:18]
Yeah. I mean, being a security practitioner first and foremost, this idea that security and compliance should be fast and cheap, it really doesn’t resonate with me. It’s an important job. Often it’s thankless, but it needs to be done and it needs to be done correctly. And with the amount of moving pieces that are going on right now, we need competent information security professionals in the space to make sure that all of our collective data stays protected. And this notion that I can buy a platform off the shelf, I don’t have to configure it, I just plug it in and everything goes green and I have this false sense of comfort that everything is working and then I can check the box and move forward, is just not right. And then also this race to the bottom of, like, pen tests for a grand or SOC 2s for a grand. This is why people are losing trust in the security and compliance space. The worst thing that could possibly happen is that these standards lose their meaning or their reputation, and then we have to go back to due diligence questionnaires for everybody. And I personally don’t want to go back to that. So what I am looking for is, if the AICPA could, I know they’re implementing things now, but it’s been a few years since they’ve done anything about it, just being able to level the playing field here. An auditor has to be of a certain caliber and can issue high quality reports that have been peer reviewed. That would really go a long way. My major problem is, not every organization is the same. They need to be treated as unique entities, and they need to put in the right controls so they protect everybody’s data.

David Forman [43:24]
Stas, I got a question for you. It’s all off the record, right?

Stas Bojoukha [43:29]
Yeah, exactly. No giant mic in front of me.

David Forman [43:35]
So Mastermind’s top competitors: Coalfire, Schellman, A-LIGN, Aprio, Baker Tilly, Sensiba, Insight Assurance, name a few here in the US. What do you think, and I don’t have this actual information, I’m just curious from your point of view, what do you think is the average age of the billable audit staff for, let’s say, SOC 2? Do you think 20s, 30s, 40s, 50s?

Stas Bojoukha [44:03]
It depends, right? If we’re talking about the senior auditors that sign off, then obviously they’re gonna be on the older crowd, but otherwise it’s young, the ones that are billing the bulk of the hours.

David Forman [44:19]
Good question. That is a really good question.

Stas Bojoukha [44:24]
It also depends on the audit firm. There generally tend to be kids, or they tend to be on the older side. That’s been my experience with it. I’d say for SOC 2 specifically, 20s for sure.

David Forman [44:35]
ISO in the US, at least, 20s. And I’m not in my 20s anymore. But at the same time, I will say this race to the bottom, as you call it, fast and cheap, it ultimately means we have to protect margins as an audit firm, therefore you’re looking for cheaper resources, hence the offshoring, nearshoring, trends we’ve seen in the past five, six years since the pandemic. With that being said, think about Dan’s earlier question around staying up to date with technology. I would also argue being fluent in technology of years past too. For example, I remember being on an audit at a prior firm, and the auditee did not use Active Directory, they used LDAP. How many of those average-aged auditors that we just talked about have any experience working with LDAP? Because if they’re in their 20s, they don’t likely have other industry experience. Maybe they don’t come from a dev side. Maybe they went through a non-traditional education route. But these very basic concepts from 10 years ago are still prevalent in some of these client environments. I mean, heck, on-prem environments, mainframes still exist. Yet everyone just assumes you’re sitting in an EC2 instance in AWS. That makes up a large part of the environments we look at for SaaS providers, but that’s not all of them. And all of them basically are pursuing SOC 2 and ISO these days. So I think there’s a huge competency gap when the benches for these audit firms primarily are sourcing junior talent. I’m not saying I have a solution for it necessarily, I understand why margins call for that, but from a standpoint of are we maintaining integrity and trust throughout the third-party assessment process? Yeah, there’s huge gaps there if we are losing that type of talent.

Stas Bojoukha [46:36]
But also just to double down on what you said, the three-day audit, the on-site audit, if you’re even lucky enough to do that nowadays. They’re not gonna know what your business does and how it generates revenue. Also, there’s a big difference between a financial firm that’s highly regulated and a mom and pop B2C shop. They have completely different requirements. And then on top of that, you throw in the technical aspect of it, and they have no idea what they’re looking at. The thing that kills me is you put all this time and effort into just something simple. Pull all the users out, and what groups are they in? Has somebody checked them over? And you drop it into an Excel sheet and they struggle with the information that they’re looking at. You have to serve it to them on a platter. And even then, can we get screenshots of this? Can we get screenshots of this? It becomes really difficult, because they’re not technical. They don’t have the experience needed to be able to understand the environment. And then you’re trying to streamline some of your own processes to make it easier, but if they don’t understand what they’re looking at, or the context in which it’s being presented, then you’re just having to go back to screenshots and collecting all this stuff manually, which is defeating the entire point. So, auditor wars. Sorry, Dan.

Dan Tangney [47:56]
Yeah, auditor wars.

David Forman [47:58]
I think if we build for the future here and try to maintain integrity throughout some of these schemes: I’m not bullish on this situation that we’re describing really getting any better in the short term. I don’t see that education pipeline happening. I don’t see audit firms, even of the size I’m talking about, forget the big four for a second, really investing back in junior talent to allow them to shadow and then slowly grow up in the space. So I would love to see that burden start having heavier weight on the platforms, right? We say, all right, the platforms are gonna really be the self policing for the technical controls here. Something that’s configurable in your AWS environment we were talking about earlier: has it changed? Did we get a flag from that in an alert? And was that actually acted on? That’s all stuff that more or less any competent professional could evaluate. Now the part that I haven’t seen yet in the platforms done really well is the governance I’ve talked about already. I think governance is a little bit more of a non-technical matter, but it’s also very elastic. It doesn’t look the same from even similar sized SaaS provider to other similar sized SaaS provider. And that’s where I think it requires a little bit more of a human touch right now, to evaluate whether or not the management system and governance is actually acting appropriately or sufficiently to the underlying audit criteria. So we start shifting our focus for these audits away from the stuff that’s more or less automated or semi-automated, and we get reasonable assurance that, hey, the visibility is correct, the shadow IT is not being missed, these platforms are configured correctly to do the continuous monitoring. Then we can spend more of that time we already have during that three day audit on the governance side, which should be the kind of missing layer here, where we say, hey, we have a new repo that popped up that is prod in GitHub. Guess what? Our CAB, our change advisory board, caught that, or procurement caught that because the GitHub invoice went up, and now we are gonna include that into the visibility of our compliance automation tool. If I saw that as an auditor, one, I can really get focused on governance now and how I train my staff, but two, now we’re taking away this giant risk in this industry of, do we have competent assessors to understand technical controls? I think they can understand them, even for an app or system that they’ve never seen before, if it’s displayed in that beautiful UX from that JSON file we talked about earlier. But at the same time, I just don’t really buy that audit firms are training junior talent appropriately, or at least in the same manner or degree of rigor that I saw early in my career at Ernst & Young. So yeah, I’ll leave it at that as the wish list.

Dan Tangney [50:50]
There you go, Dan, 10 minutes later, I answered your question. I love it. Well, we have nine minutes left and we have a few questions I wanted to toss out. Stas, coming to you first. This is from Adnan. The question was: how can small and medium companies adopt continuous compliance without expensive compliance tools? Where would you start, Stas?

Stas Bojoukha [51:15]
Yeah, that is a good question. I would run an exercise where, if you’re looking at SOC 2 or ISO, I would run through it and see, depending on the size of the organization and complexity, how much of it you can actually automate. In the sense of: does it have an API? Just touching on David’s point, some things are more difficult to automate or semi-automate, right? It’s hard to automate a pen test, you have to show up and say, you have a pen test, show that board minutes have happened. But things like, are you making sure that all your users are authenticating properly? They all have unique user names, things like that. If you are using Active Directory, hopefully not on-prem but through Azure, can you automate that? Yes, you can. Where does the evidence live? I would just do that exercise across all the controls and see how much you can actually automate based on what you have in your environment today. And then use that criteria to go and source a GRC platform. If you’re a smaller organization, there are definitely very reasonably priced GRC platforms out there that will help you with the continuous control monitoring, or the C part of GRC for sure. And then you can kind of scale it from there.

Dan Tangney [52:43]
Cool, yeah, really good. David, question for you. This came from Muhammad. Specifically written: question for David. Why do you think FedRAMP is accepting SOC 2 Type 2 certification reports as an approved alternative security framework, as a fast track pathway for FedRAMP certification, and not ISO 27001?

David Forman [53:06]
I think ISO 27001 will be added to that list at some point. I’ve had some comments publicly on SOC 2 being the first one when they came out with that RFC that Pete Waterman started circulating, I think back in January, February of this year. Ultimately, it doesn’t get you the same type of ATO that we’re thinking about here on the marketplace. It does get you FedRAMP certification now. I see ISO 27001 being added to that list. However, this is such a lightweight FedRAMP. The reason why I think they’re doing this, to answer your question, it comes down to access for the federal government. Think about how quickly we all started adopting, as consumers, OpenAI and Anthropic when they were first released. The public sector in the US was far behind that. They were not able to access that same tooling the same way the private sector was immediately. And the same thing happens across areas where there’s less innovation. So not necessarily an LLM provider, think about it from a standpoint of, they just wanna procure a vulnerability scanner. There’s like two that are approved on the FedRAMP marketplace right now. And both of those providers, I won’t name them, had to spend probably north of a million dollars US to initially get on the FedRAMP marketplace. If you think the DOD, using that as an example, is gonna reach out to them and get preferred pricing of any sort, you’re wrong. So the FedRAMP program, I have my opinions on this, again, I’m a very ISO biased guy here. It originally was designed to secure the federal government. What it’s turned into, and this is why I think we had a lot of the initiatives behind 20x, is this idea that we’ve now limited the pool of suppliers the federal government is allowed to procure from. And because of how expensive it is to even get on the marketplace, we’ve now increased the cost for the federal government while limiting their access to different providers. And I think that was a negative consequence of the program. I think that’s in part why the Department of War has come out with their most recent statement around CMMC Level 2, and apparently it affects more than just Level 2 as well. And this current pause we’re in, I think there’s a balance here around securing the federal government but also allowing the government to innovate and act in a way where they can stay up to date with modern technologies and the latest and greatest platforms, and honestly, LLM models as well.

Dan Tangney [55:37]
Very cool. David, while you’re in the seat, that was great, nice work. I got another one for you, from AJ. The question is: what are the best practices for managing exclusions and exceptions from your continuous monitoring program? I’m curious how you prevent people from exempting multiple samples, transparency over what was excluded.

David Forman [55:59]
AJ, we have enough one-on-ones, you could have just asked us in person. But I’m sure others were wondering too, AJ. How would I answer this one? This is kind of funny. So we were going down a different talk track earlier when we were discussing here with Stas, this idea of evidence provided by client, PBC, provided by client, that’s what we called it at A-LIGN when I was over there. It’s this idea that the auditors are not actually in your systems, hands on keyboard, pulling the evidence and extracting it themselves. Very rarely is that the case anymore. There’s a few schemes that have a little bit of requirements around there, but the days are, I’ll say decades ago now, when an auditor would actually show up at your data center and literally plug into the terminal and start downloading stuff and getting those screenshots themselves. Long over. And now it’s like, what do you need to see? I’m not sure I can give that to you. What do you need to see? That’s the auditee now, right? And so this idea that AJ is bringing up is: how do we make sure that the evidence being provided by the client is actually representative of the sample we’re trying to take of the population? It’s a huge issue. And you’ll actually see variation from audit firms on this for the same scheme, so SOC 2 or ISO 27001. There are some audit firms out there that I’m aware of, they collect no evidence. Instead, they do it all as live walkthroughs. I’ll shoulder-surf you, Dan, the entire time while you share your screen. You’re an admin on your AWS console, your Azure tenant, and we’ll walk through the exact tests I wanna do together. And I don’t need to get screenshots, I’m looking at it live. So that’s one option to kind of get around that, especially if you’re suspecting nefarious activity with your auditee, which does happen. But also, I’ll be honest, it’s interesting: when these second-party internal audit functions of these companies, especially mid-market enterprise, start getting very mature, they start becoming almost like the auditee as well. And so now you run into the situation where they’re kind of protecting their own job when they don’t wanna show you exceptions that haven’t been acted on. So I would say get down to your version of root access for the alerting tool, and you can find all the data you want. And then you can start doing that same triage of whether or not we’re actually going through certain remediation activities on a timely basis, based on whatever alerting policy you’ve set up. That’s probably the long answer.

Dan Tangney [58:20]
Yep, good. Very cool. One minute left. Do you both get time for a shameless plug? Stas, if anyone wanted to see Compyl, learn what we’re doing, what’s the best way to do it?

Stas Bojoukha [58:29]
Go to compyl.com, take a look. If you wanna see a demo, sign up there. Demo request right on the website.

Dan Tangney [58:34]
Very cool. David, if people wanna learn more about your services, your classes that you offer, what’s the best way to get a hold of you?

David Forman [58:40]
Yeah, our website is mastermindassurance.com. You can also find us on LinkedIn. We are LinkedIn fans, unfortunately. So you’ll find us on there, and DM me as well, please connect with me. And then a shameless plug here, because it’s the Shameless Plug Minute: if you work with Mastermind, we are not average age in our 20s. So a lot more competency over here.

Dan Tangney [59:04]
There you go. David, I’m gonna put you on the spot before we end. Next time I see you in person, can we do a Compyl for Mastermind polo swap? Kind of like the pros do.

David Forman [59:16]
Oh yeah, that’s a great idea. Can we do the swap for the polos? Do you think they’re cool enough for Peter Millar? Yeah, we can do that. That sounds great.

Dan Tangney [59:23]
We really appreciate it, David. Thank you, Stas. Appreciate you guys being here. Everyone that stayed on for the webinar, this will be sent out to everyone. Anyone that registered that didn’t make it will also get a copy of it as well. Appreciate both of you guys. Have a great day.

David Forman [59:38]
Thank you guys. Thanks, everybody. Bye.