Cloud Crunch
Cloud Crunch

Episode · 1 month ago

S3E2: How to Create Business Intelligence before Cloud Migration

ABOUT THIS EPISODE

Last week on Cloud Crunch, we covered, "Dont Leave Out Your Data & Analytics Element when Migrating to the Cloud." Today we cover, "What is Business Intelligence" when migrating to the cloud. We are joined with our new co-host Fred Bliss, CTO of all things data at 2nd Watch and our honored guest for the second time, Evi Hatzopoulos.

Involve, solve, evolve. Welcome to cloud crunch, the podcast for any large enterprise planning on moving to or is in the midst of moving to, the cloud, hosted by the cloud computing experts from Second Watch, Michael Elliott, executive director of marketing, and Fred Bliss, CTO of all things data at Second Watch and now here, are your hosts of cloud crunch. Welcome back to a new season of cloud crunch. Now, in this season we intend to address many of the difficulties and the opportunities of your evolution in the cloud. We'll hit upon topics covering application modernization, enabling cloud native development, data insight and cloud economics. Now, normally I'd be joined this season with the ever present, ever learning skip Barry. However, he's still on holiday, so sitting in the code seat today is Fred Bliss, CTO of all things data at Second Watch. Greatly appreciate you sitting in that code driver's seat today. So welcome Fred. Thanks Michael, happy to be here, and I'm joined by my favorite coworker, heavy Hotspolis, who I've had the pleasure of working with for the last five years. Here we go. Thanks for joining morning, guys. So favorite coworker. That's not gonna get you in trouble back of the office, is it not? Probably for the US, and I do need to say I mean it is International Women's Day heavy, so we expect great things from you today. Oh, I will make my women proud. I love it. Think about my daughters, you know, she's getting ready go to college. So it's a great time to celebrate women in general, especially women and stem. Yeah, did you know where she wants to go? We're working on it. It's a tough one. It's a hard decision to make at like seventeen. So I don't remember that. But Anyway, let's move forward. So in our last episode we covered kind of don't leave out your data and analytics elements with migrating to the clouds. So let's kind of continue on with that theme. So in today's episode I want to kind of cover how to create business intelligence. So I always want to kind of start off with what is business intelligence? It's kind of funny the word of business intelligence has taken on a different meaning over the years. You know, I think originally we're back in the nineties, it was called decision support system back when I was in school. That's the textbook I was learning from, became business intelligence, and now I think the hot word is analytics. Right. So when you think about Bi analytics, decision support systems, they're all kind of the same thing. But it's really about taking your data, making sense of it and getting some kind of business value out of it, and there's a lot of ways to do that. There's a famous bi maturity curve that I don't necessarily agree with anymore, which is, you know, you start with the what happened, then you go the descriptive, then you get to the diagnostic, which is why did it happen, and then you get to predictive. And I think there's a sentiment among organizations that you have to hit each stage along that curve before you can move to the next one, and I think what we're seeing now a little bit is you don't necessarily have to. Sometimes you can start with the diagnostic, sometimes you can start with the predictive. I don't know if you're seeing the same thing of it. Yeah, I mean I think the main point of the is just end users that aren't hands on Keyboard, you know, interacted with the data often to see how a sense of kind of what's happening from the organization and there's not really a bad time for them to get eyes on what's going on right well, it's interesting, though, is the more eyes you get kind of on these data sets that were visualizing, the more interest there is in the data itself. So I would agree. I would say kind of throughout the whole journey, there's not really a bad time to introduce B I and are you seeing kind of the same trends? I'm seeing that I feel like ten years ago a lot of the reports that were created were largely driven by it and it was this big centralized data warehouse controlled by I T.

...centralized reports were created distributed to business users, and a lot of the business users all they were kind of doing was saying that doesn't look right. Yep, that's right, and it wasn't really stent analyzing the why behind the data. It was more looking at the numbers and making sure that they were actually what they thought they were, and that was kind of it. Yeah, I think now the getting kind of the business users involved with the design of the dashboards and also kind of understanding the business rules that are going into it, because it did used to take a long time and I think it was a really stressful process, both for I t and the end user, because they'd want to kind of get insights on what's going on and then, you know, the dashboard that was delivered to them. Maybe it wasn't the metrics they were expecting to see or kind of things weren't moving along as they thought. So then you have to work backwards and understand, okay, well, what filters went into this and what business logic went into this and what data sets are remissing and okay, you know, you're doing a bunch of stuff kind of silent over here. We don't have eyes on that. So I think, you know, what's been cool is we're seeing a trend where the business is getting way more involved with I t to kind of build out what we want to expose and what data since we want to use. And it's really mimicking the business operations versus I t s idea of you know, what that business logic and what the business operations are. So there's a shift going on between I t driving this and the individual business unit driving this. Yeah, and I think we've seen this change a lot and I think when I think to our most successful projects, it's a it's a partnership between business and I. You know, and again I think back to ten years ago or, you know, even longer when, Um, it was, like you said, it took a long time to create these things. Sometimes it was a one to two year project and it was done in the traditional waterfall STLC. Let's go gather requirements from the business, let's go design it, let's go build it, let's go deploy it. Here you go, business users, here's your reports, Um. But by that time that happened, Um, like that. They they either it wasn't what they expected, or maybe people turned around, or maybe the business change and it's just not a great way of doing of doing analytics. Because, Um, I think business users need to see what's possible first, right, and that's why this kind of iterative, more agile approach towards building the I and analytics platforms has been way more successful than it was in the past. I was going to touch on that too, about the operative approach. I feel like maybe in the past, like you said, front all the requirements and to begathered up front and then, like you unveiled this massive dashboard that was supposed to be, you know, the best thing ever, and now at least. What I'm saying is will kind of roll out a dashboard with some functionality and we'll let users kind of play with it and see what they like what they don't like, and then we'll just keep rolling out additional features on that same dashboard. So it kind of grows with, you know, what their needs are, um, and they're loving it. You know, they get excited for these rollouts. Were not waiting months and months and months or a new dashboard. It's kind of what you're used to and we're tweaking in a little bit. Or if you know what kind of spins off into an additional dashboard, then they're happy about that too. But kind of keeping them engaged and keeping them involved in and what I t is doing. Um, I think they like it because it's also like their voices being heard. Right at the end of the day, these dashboards are supposed to service them and kind of you help them see what's going on, wanting to change what's you know what's going really well. Um. So I think the more involved they are, uh, the happier. Probably I t is too, because the less complaints and at how reports they got. So kind of benefit's both sides. So let me ask you this. WHO's now paying for this information? Is it still I t that driving and that you're having a conversation with actually financing these business intelligence dashboards? Or is marketing? Is Finance, is sales? Are these other departments starting to be more in the driver's seat around this is the business intelligence we want and coming to you and hopefully partnering with I t, versus I t driving it? I think I very much depends on the on the B I...

...maturity of a given organization. When, Um, when you look at your analytics platform or your data platform as an I t product, one of the first thoughts that come to mind, um from CEOS is how do I cut the costs? How do I save costs on this? I'm spending so much money on this thing. Um. But when you've got it driven by the CFO UH and you're starting with financial data and making it easier for folks to be able to dig into the transactions and tie the General Ledger to your forecasts and run predictive algorithms on top of that and get a better forecast out of it, Um, you start to see business, individual business units paying for some of the cost of doing some of these bigger projects. And you know, they also don't have to be these big one to two your engagements anymore. You can start with a very simple use case, um, you know, get a couple of business users involved and build something very skinny but end end and use that to expand upon over time. I think she once you kind of start involving different business units, it's important to think through your tool selection because different departments are going to respond differently to, you know, the dashboard. Maybe the finance department wants something more like a sigma, where it's kind of excel us and you know, it's really easy for them to grasp, you know what what's being told and they can download it and kind of their own manipulations. But it's off of a trusted data set. So it's it's kind of like a familiar interaction with the tool, but the data behind it is kind of the centralized data warehouse versus something that they're just exporting or extracting or kind of doing on their own computer. Um. So yeah, so I think maybe what works really well if you're starting with, Um, you know, the marketing department. Maybe they want more visuals and then when you're, you know, trying to get finance on board, you can think through and get a little bit more creative about, okay, what's really going to resonate with them. So we're getting everyone across the organization really excited and really invested in what we're doing. And I think one trend I've started to see is Um. You know, again, ten plus years ago, uh, you would centralize around one B I tool and that was it and you had a lot of the business logic and code going into this B I tool. I think you know business objects and Um and O B I e and Cognos. Uh, these were big, lofty systems that were created and if you wanted to access reporting of any kind you had to do it through this Um. And I think, like you said, uh, this is changing a little bit. There's a lot of innovation happening Um, you know, in the data platform space. You know, we see that with products like snowflake. We see it with DBT. Around the data engineering piece. We're seeing a ton of new innovation and I think the analytics site is what's going to come next. You know, I think to Um. You know a company called C Su and they focus on the diagnostics. So the hy Um, and that's really interesting, you know. You know, when you think about the context of covid now, Um, you know, retailers in particular are dealing with supply chain issues, right. You know, this is a common problem. Everyone knows about it. Um. And if you look at some of the research, who do uh, customers, I guess, blame or feel is responsible for when an order is late or it never arrives or, Um, you know, it's back ordered for the next couple of months. It's typically the retailer that takes the brunt of this. So, Um, you know, C sue kind of UH looks to take all this data you have and tell you why things happen. So why was this shipment late? Why are these shipments here late? Um, and then, more importantly, what can we do about it? So how can we increase the loyalty of this customer who was ordered we just were never able to ship out? And you can start to use this big data to get to the why, more so than looking at the numbers and seeing why. You know how many returns we're having and trying to figure out what to do from there. And it's a lot faster too, because if someone were to invest that time to kind of understand why things are moving up or down when it's not, you know, extremely logically. I forget the example they used, but it was something about military bases and Samsung phone. Yeah, there's a great one where...

...they're trying to essentially figure out why certain geographies are not upgrading their phones and it turns out that they weren't marketing or advertising to places where there were military bases which were strong buyers of the brand. And so combing through that data we see a lot of this happened kind of organically. It's like you said, you're looking at DASHBOARDS, you're digging into the details. You might create a permutation of that dashboard and start to dig into the data. You might bring into excel, but that's a lot of analyst work and I think what some of these tools, like season now we're doing is to almost empower the analysts to be a little bit more Um, more effective at what they're doing, to be able to find these trends, why things are happening, and then bubble that up so that business leaders can make better decisions on how to act on this more quickly. So I want to go back to something you said, evy, around thin data sets. So is it one data warehouse, data leake? Help me with the terminology and and are there issues with creating a thin one when you need to start that expansion as other departments and other data sources come online? I think important thing is just to kind of start right. So, even if the data that you have in your your data warehouse is kind of skinny and it's a small data set, I think still communicating that that is the data that we're working off of and then being very open to requests. So if there's, you know, a business unit or a person that has been running his own report off of a system and he really likes it and hit all of his fancy business logic is in there, that's fantastic, but like, let us bring that into the data warehouse that we're building, you know. So I think it's not about like telling people to stop doing their independent work. It's more so, Hey, let's we love what you're doing here and you've done a lot of the runt work and it works and if you like it, probably ten other people in your role will also like it. So let's just find a way to move that into our environment. And that way you kind of start like organically expanding your data warehouse, right, so that it's more valuable to more people. Um, but I think, yeah, the important thing is just to kind of get started with that initiative. Yeah, and I've got a great example around that is you see this a lot with data science team in that they're building kind of parallel data pipelines to the data warehouse initiatives. And there's a reason for this. Right. Um, not every data warehouse is going to have every data point that they need. But if you bring the two together and Um, you've got data science teams now taking the data, like you said, from a centralized, trusted place where that data exists today, and augment that with the data that they're missing. This might be external data or just some raw data at a very fine grain that that the data warehouse hasn't been modeled for yet. Um, you can start to use this trusted data, Um, and have your own pipelines running off of it. And we see this at a lot of really successful organizations that are a little bit more mature on the analytics side, where they've got data analysts embedded in within the business, and so it's not necessarily a matter of weight for the data engineering team to go build it in the data warehouse. But when there is trusted source of data, so say the trusted source of what a customer record is, you don't have to go reinvent the wheel. You can take it from there. And you know, I think what helps drive a lot of this is, again, you know, better products that are coming out that do different things. Um, thoughts about us a great example every I know you you've seen that as well where, Um, you know, from a data discovery standpoint, I've got a customer right now where they've got so much data out there that business users don't even know what's out there and what's been built. And with thoughts about they're able to plug it in instart, just kind of looking for it, almost like a data catalog in a way. Yeah, thoughts, that's pretty cool. If I remember correctly, it lets you like tag certain elements of Your Business. So if you know across the organization you use three different words really mean the same thing, if you tag all those things the same way, you can just search that one word and then all the data pertaining to those three individual things kind of pop up. Um,...

...so it's a lot. It's a much faster way, kind of to what we were saying about s Su two, like you could do that. You know, an analyst can sit there and do that manually. But these tools really really accelerate that process, um and it's just a matter of knowing when to bring them in. And I think once the data is at a place like Fred side, where it's trusted, Um, then you kind of empower people to say, okay, these are the tools that were, you know, looking into or written demos. Who want to bring these guys in? And this is kind of what it would allow different units to do. And, like I said earlier, it's like, depending on the department, maybe some gravitate towards one more so than the other. That's totally fine because you know, at the end of the day, the underlying data that is consistent and it's the same Um. So it kind of allows you to keep a lineage and then, you know, empower people to do cool, cool things that they you know, they want to visualize it date in a certain way. They're they're able to do that and I think the commonality there is that, Um uh, you have to build your logic inside of your data platform. Right, not captured within these individual business intelligence tools, because they will there will be others coming at some point. Someone in some business unit will bring a different tool. Well, that brings me to my next question. You know, you're talking about tools and applications. Once, once you've built this, how self sufficient are clients to be able to then evolve it? Great Question. I feel like that kind of depends on training. Right. Like some of these tools, I mean you you need to know kind of how how they operate, Um, and you what you don't want to accidentally do is manipulated so much that it no longer is showing what you intended it to show, because you know you can go down a rabbit hole with a lot of these tools. So I think proper training is important. Number one thing. need to understand the underlying data set, right. That's the most important piece here. What data is it comprised of? What is it missing? Um, just so they don't accidentally think, okay, well, this number is a hundred and all of these factors go into it, when in reality maybe they don't. So I think having a really good understanding of the underlying data set's important and then understanding, you know, the best way to visualize that data in these respective tools so they can go off and kind of build their own reporting while it's still a consistent message. That's kind of tough because you want to empower self service reporting, but I think, you know, if you if it's a free for all, it can kind of take a bad turn and I think, I think it's a matter of knowing your audience too. Um. You know, during data strategy projects we are able to, you know, pretty quickly figure out what type of organization there are, some of the change management that needs to happen and what's going to be successful and what isn't. Um. You know, I remember a client from a long time ago where, you know, it was going a one year data warehouse project and they rolled out a big Bi tool, enterprise B I tool, and, uh, the mantra was, I t is not going to create reports. Um, everyone in business is going to do that. And so when you're asking, Um, a sales rep of a manufacturing company to go create their own reports off of data using a tool that Um, you know, it's even, like vy said, a little hard for some more power users to understand, you're setting yourself up for failure and that's kind of a you know, the building and they will come not being true. But uh, if you think about some organizations as being a mix where you've got some, you know, the majority of business users, when they look at a dashboard, the most they're likely going to do is look at it very quickly, just like you know, Um, you know we would. Uh, you know, I think about it almost like recruiting with the resumes. You glance over, you look at it, you get the information you need and you move on. Um, that's gonna be maybe, maybe, sometimes more of the business but UM, they're gonna be the ones who can dig in and try to uncover some of these insights that are in that data. And again, I think some of the tooling that's going to come out and some of the innovation there is going to continue to chip away at that to make that number kind of become a little more balanced. Yeah, and I'm probably a little bit like that user you describe where I go to Beth on the Second Watch team is I'm afraid of breaking something. So it's given...

...me of what I need. I just need that a little bit more. I won't touch it all. That best help build that for me. Yeah, and there's nothing wrong with that and that's why I think you know, when you think about organizations and the power of a business analyst or data analysts, I don't know about you guys, but I've seen the business analyst role be almost lessened to a degree, right like there used to be business analysts kind of all over the place at organizations and now, you know, I think that the trend is coming back a little bit where we're seeing the value that bas and data analysts bring, but they are absolutely invaluable embedded in business units because they can do kind of those things that you were talking about, Michael, and dig into the data and try to figure things out and then bubble that up to you so that you can make the next decision. So last question for me. Where does artificial intelligence and or machine learning fit into really extracting that business value? I think it's in a couple of ways. One, we're finally at a play where I think it's not just kind of a niche fad. I remember one organization a couple of years ago that went down an AI project simply because there was a mandate from the top that we need to do ai. That was it. There is no business these case. But there's real problems being solved today using machine learning and there's things getting into production. But, you know, I think the hardest part of this is getting these models into production. We've got all the data, organizations have the talent. It's building up to certain degrees. We're getting the tooling in place and now the question is more around what use case do we go after first, and when do we do it, and how do we get it into production, and then how do we sufficiently explain it so that we can give the organization trust that what this model is saying is accurate? And I think you're seeing some tooling out there as well. Again, you know, going back to s, so they kind of combined the to write the diagnostic it's a combination of running ml and sequel queries together over and over and over again again looking for patterns. So if your loyalty rate, or you're returning customer rate has gone down over the last couple of days and your number of, let's say, late orders has also gone up, the next question is why, and machine learning can help you kind of figure out the reasons for that, not necessarily. You know, a couple of years ago what we used to see is I want to know the present likelihood that this sales order or that this sales opportunity is going to close and be one. Now it's more what are the attributes associated with this one being more likely to win or more likely to lose the any final thoughts on that? I think with AI and machine learning in general, the roadmap is really important. A lot of clients obviously want to jump to predicting what's going to happen with a certain use case or metric without really understanding what has happened historically in the past. And you don't necessarily need to do one before the other, but I think it kind of just helps your data get to a bit more of a mature state in order to really take full advantage of the data science component. So a lot of times like we'll have these clients, and you know I have one in mind right now, and I know they have a bunch of use cases and a bunch of metrics that they kind of want to get a grasp on for, you know, this year, and almost all of them have kind of a machine learning component to them, and our recommendation is, I know we want to get there, but let's do these you know, three things first before we can get to that component. That's not going to be true for every organization and, like Fred side, there's tools that are kind of enabling a little bit of that sooner. But I think when you're thinking about machine learning that road may have to kind of understand what am I trying to get out of this? What answer do I need by the end of this is really important order for that to be successful. That's absolutely spot on, excellent. Well, I want to thank you, Fred and heavy for joining us today to kind of go over and discuss how to create business intelligence. Any find words of advice to companies looking to extract B I out of their data? Start Small, start very small right. Find a use case that's difficult to solve in your old today, whether you have a data warehouse and...

...you have bi tools. What's not working today? What takes a long time? And get moving right and iterate. Yeah, and also, if you want to kind of gain some excitement around the organization, people are visual right, so that's probably the best way to get them excited about what you're trying to do on the data front as well. Awesome. Well, Happy International Women's Day to our audience. Thank you for listening to our show. This podcast is intended to add value to any large enterprise that is planning on or moving to or it's currently focused on leveraging the value of the cloud. Send your comments or suggestions to cloud crunch at Second Watch DOT com. Um Michael Elliott. Have a good day. You've been listening to cloud crunch with Michael Elliott and Fred Bliss. For more information, check out the blog second watch dot com, forward slash cloud dash blog, or reach out to second watch on twitter and Linkedin.

In-Stream Audio Search

NEW

Search across all episodes within this podcast

Episodes (33)