Cloud Crunch
Cloud Crunch

Episode · 3 months ago

S3E3: How Snowflake Empowers You To Unleash Your Business

ABOUT THIS EPISODE

Welcome back to Cloud Crunch. Today's episode, we will cover "How Snowflake Empowers You to Unleash Your Business". Snowflake, is a worldwide leader in cloud-computing and offers a cloud-based data storage and analytics service, generally termed "data-as-a-service. "This powerful conversation is led by our CTO of all things data at 2nd Watch Fred Bliss, and Director of Marketing Michael Elliott.

Involve, solve, evolve. Welcome to cloud crunch, the podcast for any large enterprise planning on moving to, or is in the midst of moving to, the cloud, hosted by the cloud computing experts. From Second Watch, Michael Elliott, executive director of marketing, and Fred Bliss, CTO of all things data. At Second Watch and now here, are your hosts of cloud crunch. Welcome back to a new season of cloud crunch. In this season, we intend to address many of the difficulties and the opportunities of your evolution in the cloud. We'll hit upon topics covering application modernization, enabling cloud native development, data insights and cloud economics. I'm joined this season by new co host Fred Bliss, Cteo of all things data. A second watch and I'm looking forward to this season. Fred, lots of cool things to cover as we continue to see the evolution of the cloud. Thanks, Michael. Looking forward to it. Welcome back to cloud crudge. In today's episode we're going to cover how snowflake empowers you to unleash your business. Now, joining me as always, just Fred Bliss, CTO of all things day to hear. At Second Watch. So, Fred, how snowflake empowers you to unleash your business. That's a pretty strong statement. Do you think we're gonna be able to back that up? Yeah, and I think it's important to remember that. You know, snowflake, it's an ecosystem right by itself. It enables quite a bit that businesses weren't able to before and I think it's created standards for things, you know, in the data and antaletics ecosystem in general across enterprises that have almost become standards and things we take for granted today, you know, right. You know, if I think back to, you know, six years ago when we first started partnering and recommending Snowflake, Back when they were, you know, essentially a database data our house start up out of out of the West Coast, and I remember saying, you know, this is really cool technology, but I don't see how enterprises are going to, you know, adopt this right. And this is at the time when sequel server is strong, oracle is, you know, kicking butt everywhere, Terro data is still on a roll, and to think that this browser based cloud within a cloud is going to take off and become kind of the de facto data cloud for enterprises just, you know, it was completely beyond anything that I ever thought but at the time and I was like, okay, let's try, let's see what happens, let see how it works out. But it's created some things that just make things easier that we couldn't do before, you know, being able to bring in Jason Data and semi structured data. We've got one client right now that's bringing in all their weblog data, as it happened, so as they're clicking on their website and their customers are clicking on their website, that data is being streamed directly into snowflake, which is then being streamed into the B I tool that they're using, and so business users can watch in real time as their clients are hitting the website in...

...various pieces of it and start to understand what are people doing, what are they looking for to what do they want, what are the gaps, and tying that all in with everything else across the enterprise. It just wasn't something you could do before. Right, I still remember the days of writing these big etl scripts in sequel server and you kick it off and you wouldn't see if it actually loaded successfully until twelve hours later. I remember when I first started my job. They essentially used Informatica and it was I think the job ran for about twenty three hours and this big enterprise and if it didn't finish at that twenty three mark, then you would essentially restart and you'd have the next job kind of overrunning. So they had people whose sole job it was at night to sit there and watch to make sure this job was still running and to kick it off again from where it failed if it didn't. I mean, you don't want to go back to when I was developer. I was writing code into database and the reality is that database probably still exists. They're probably still using it. So how absolutely, and it's it's become a source system now for a lot of these snowflakes, you know, data platforms that we're building. Well, it's kind of it's talk about it. You know, where does snowflake and that capability fit into taking these legacy kind of systems that you can't do a lot with? How is snowflake changing that? You know, I think something that we've always told our clients as they're thinking about what systems and, you know, what technology to buy, right. That's always, you know, the biggest question from the technology side. The business side tends to be. You know, I don't care, just make it work, depending on the size of the enterprise. But I think a lot of the early market immature was around performance, right, like they can load terabytes and terabytes of data and that's just something you can't do in a typical sequel server environment or a typical oracle environment. For us, though, it was more about the speed to insight, so being able to do things, Um, you know, like get all your raw data and really really quickly and then throw a view on top of it and start querying it within, you know, maybe a couple of days, you know, maybe even a couple of weeks. You can have something end to end. Now it's not going to be, you know, cost optimized, it's not going to be Um, it's not gonna be perfect, it's not going to be your end production state, but you can start to look at something and get an idea of, you know, this is what's possible, this is what my data is telling me, this is what I can do in a couple of weeks, as opposed to Um, you know. Again, back in the day it was three to six months before you even new. If the business us as were getting what they wanted out of it, sometimes even longer. So time to solution is a huge component of this um where you went from months to days to understand the value of that data. I think what it's done for for us as consultants and builders is that if we were to ake...

Um, you know, a snowflake environment, versus Um, you know, again the typical sequel server on Prem Environment. You know, and we've had that before, trade offs of should I buy it or, you know, I've already got a sequel server on Prem should I just use it? And the difference for us is we can do it faster in snowflake than we can in a sequel server environment. And the other side of that coin is when we deliver that to our customers Um, it's going to be less time on their side to support and maintain it. Now, yes, are they going to pay a little bit more than they would for a sequel server that they already own and that they already have a license for? Yeah, they'll pay more in consumption costs for it, but the cost of having to hire someone to manage those indexes, take care of backups, make sure everything is running smoothly. Things aren't down. Um. It's the simplification of the data and analytics ecosystem that I think snowflake brings. Um that even some of its competitors don't write. And but there's trade offs, right. It's it's not perfect at everything, Um, but it brings, I think, simplicity. Well, I mean, and it doesn't fit for every every client or every application. I mean if you're in a traditional or CO application or sap Hanna application, you don't just lift and shift to a snowflake. So what we're we're where do these legacy applications running on these legacy and data systems? What are you able to move over? What are those case studies that really work on snowflake? I think the best use cases are going to be both your semi structure data and your traditional structure data. And you know now that snufflakes started to support unstructured data. Um, you know, even bringing in audio files. So being able to bring in, uh, you know, your call centered log data and have the audio file attached to that and be able to run a transcription service off, you know, a ws or as are or G C P and then load that transcription right into that same record. All these things are possible now and bringing in Semi Structure Data and Queer it in real time. These were just things you couldn't do before. Right now, Um, some of the tradeoffs are that, because of the way snowflakes architected. Um, if you need data in, you know, perfect real time, where you know it's I'm hitting an Iot device and I need to see down to the second, you know what's happening with, you know, say, some manufacturing device, and these are critical business decisions that need to happen on the second or millisecond basis. Stuff Lake is not going to do that right. It's there's always gonna be some latency involved in their systems that are developed for that kind of thing. But you know, I think that you know being able to bring a sap Hanna data. You know, we do that today right, Um, it doesn't really matter what the source system is, we can bring it all together and match it all together. The hard work is still the same as it was twenty years ago, regardless of technology. It's understanding the business rules and how how the customer wants to see everything. But the UM, the tradeoff is versus, you know, some of them petitors, is it's you know it's obviously proprietary system Um versus some of...

...the other ones out there that are more open where you can bring your own tools. But I kind of uh, I guess compare it to just like when you're going with the aws native services, like if you're using lambda services or uh, their own container services, all these abstracted past services. Your paying for proprietary services but you're getting UM abstraction and ease of use out of it. Right. supportability goes away a little bit, but the trade off is it's a little bit easier to use and build for. Let's talk real quickly about the business intelligence. You're able to pull off a snowflake built system versus more of a legacy architecture. Are there distinctions? Are Their differences, and I think it Um. It mostly comes down to Um, how you're doing the queries and how much you're doing Um. It was something like snowflake. Right, if you've got Um, you know, if you're under a terabyte of data, and we're not talking huge amounts of data, you're probably not going to see a huge difference in the performance using Um. Let's just take any bi tool hitting a sequel server. While uh well modeled data warehouse versus a snowflake one right, especially for if you're just looking at let's say summary Data, aggregate data, but when you're getting into Um, you know, millions of rows. Let's say I want to see you the last couple of years at an aggregate level and I want to be able to quickly drill into those details down to the individual transaction level. This is something one of our clients we're trying to do. Before where Um with their sequel server environment and Azure, they had to load everything into power by, into memory, and you couldn't fit in all the transactions right because all these transactions over the over the last couple of years, it's a lot of data to fit into memory for the way power by works. So if any one wanted to understand, you know, individual transactions or look for discrepancies or kind of wanted to dig into a piece of the General Ledger, they couldn't do that before Um. With snowflake, now we can bring in some of that aggregate data into memory and still get that speedy performance. But now if they want to drill into details of data from they can drill into it quickly and a live querry is going to hit snowflake and bring that back in a really quick period of time. So to the end users you've basically opened up their entire data set, their entire data ecosystem, which is something you just couldn't do easily before you could do it. It was just a lot more work. USABILITY of snowflake versus having to understand a s and behind a system or red shift or is there any distinction there? I think there's some minor ones, right. And if you know, if if we compare Um red shift and UH in Snowflake, I think snowflakes embraced sequel and kind of brought sequel to the forefront...

...again, which is great for Um, you know, for data for data professionals right, and for for anyone that's been writing data analytics solutions for a long time, because there was this period of time where hudup was coming in and was kind of making sequel almost feel like a second class citizen. And there's a reason hudup didn't get largely Um, you know, widely adopted. But when it comes to redshift and Snowflake, you know, sequel is still king for both of those. Are some minor differences between the two. Um, I think there's differences in the way you architect things. With red shift you have to be, Um, very much aware of how much this space you're using, how you're partitioning things and making it sure that performance is always on the forefront of of how you build your solutions. Snowflake, of course it's not just gonna be instantly plug and play, but it's a lot closer. You don't have to really worry about, Um, certain performance turning things until you get to a certain scale, although things like how you order your data still matter, right, like all those bay sick things about building good, well architected data solutions. They still matter, you know, building star Schemas. That still matters. Um. When you get into things like data bricks, um, that's a much different system, right. The architecture of a data bricks is very different from the architecture of a red shift or a snufflick. Now snufflick calls itself a data cloud, and is that part of that distinction you talk about, or can you just kind of help the audience understand what? What is the data cloud? What does that mean? Yeah, I think it's I think it's been an evolution, right. You know, when we first started working with them, the only use case was a data warehouse, and now their data applications, there's things like data sharing Um. Again, it's all part of these standards that Um that have just become almost commonplace right when, I think, when you think data sharing, this didn't really exist before. Stuff like there was. You know, most companies are still using FTP to share data around with different partners and, you know, even internally, and snowflake kind of opened the door too, you can have your live data with your partners and even have these data clean rooms where you can have multiple partners, you know, all working off the same data with security applied, and be able to contribute and add your own. And when you think about external data too, there's so much of it out there. The hardest part with getting external data, for for clients, is how do I get it and what do I get and what's out there and what's good now, and I look back of you know, there was all this external data, but there was no way to bring that data in. There was no way to bring that data in where there was a trust factor around that. So that is definitely an evolution that has occurred. I mean API development started to allow some of that, but there was still there was no standard right there. And there's a lot of platforms popping up now that, um, that are kind of data curation places where you can have a trusted...

API, you can pay for it. But I think with snowflake it makes it easy, right. You Click a button, you get access to the share, you pay for it if there's a cost to it, and then you can query it instantly and if it doesn't feed your use case, you stop using it. Exactly all right, what are some of the considerations we should think about when adopting snowflake, just to keep your platform extensible and flexible? I think the key thing is again to remember that, um, even though the technology has gotten better, it doesn't mean that you can throw away good design and good architecture. Um. You know, if we think back to the way, you know, things were modeled again in the early two thousand's, it was kind of an inman versus kimball debate, right, and I think you know kimball is is kind of went out my opinion, uh, probably a lot of others. And so that's the traditional, you know, dims and facts star Schema. Now do you need to build it exact actually the way the Kimble book that was originally written, the data wherehas tool kits says to do it now. Um, there's some shortcuts you can take now. You don't need to do every little performance enhancing thing to get every little piece of CPU out of out of your system. But, Um, you know, I think it's less about the technology, Um, you know, whether it's on snowflake or any other Um, any other solution, and more about thinking big picture. For Uh, what data are you bringing in? How are you going to build this in a way that you can bring in future data sources that you don't have today? How can you quickly change business rules so that, uh, you know, if if a core piece of the business changes or you acquire a new company and you need to change the way that your code works, that you don't have to go change it in twenty different places? What happens if you change your bi tool, or what happens if if your company decides to adopt a couple different bi tools? Or once you start building applications on top of your data, they you've already built? These are all considerations right that you that I don't think it's really changed with the technology but, um, there's a lot more components to it than there were before. You know, back in the day it was via database, by a detail tool, by a bi tool, and you're done. Now there's all it's it's less about, you know, how do I build this and more? How do these things connect to each other and talk to each other, and how can I make sure that I'm in the future for this in the right way? Now, a lot of what you just talked about, though, is consumption, consumption, consumption. How can I consume more? How can I consume more and more and more? How do you keep, you know, that consumption cost in line, because in the cloud we love you when you when you consume more. Yeah, and it's funny because I think those key things are also a big part of the cost control. You know, we've we've done a lot of projects where we've come in and we've seen what prior developers did in their snowflake environment, and a lot of it was kind of thrown away then, except that we you know...

...we talked about before, which is well architected and well modeled systems. Yes, you can throw all of your raw data out there in any format you want in these big, wide tables that have tons and tons of rows in it, but man, it's going to be inefficient and you're gonna have to scale up to get any kind of performance out of it. You build this in a way that you know has well architected data models, it's going to make a huge difference in your performance. You know. Aside from that, I think some of the big things are that, you know, we tell our clients are data loads. Right, how often do you really need to Load Your General Ledger? Yes, would be great to get it loaded in real time, but do you really need it that way and what's the cost worth it to you? Right? And you know, we had one client who wanted to essentially load their general ledger in every fifteen minutes versus every four hours, and the differential cost of doing that every year was about four dollars. But you know, being able to look at that and say this is the cost of doing that, is it worth it too? At least they can make that in for decision. Right, for some data sets it will make sense to do that. For the General Ledger, maybe not. I every four hours seems a little extensive to me as well, but yea. So how do snowflake data cloud equip retailers, consumer packaged goods to really stay on top of changes and address those pain points and, ultimate guess, Meat Business Needs. I think the evolution of these industry clouds that they're developing shows that they're thinking less about the technology now and more about how do we enable these individual verticals to be more successful. You know, we see that with healthcare. We're seeing a big push with retail and I think it's it's again helping customers find again the right data sets, the right external ones, being able to connect with all the different partners through data sharing. And you know, eventually I think we're going to start seeing more and more industry built, purpose built solutions built on top of snowflake. You know, snowpark came out, I want to say, about a year and half ago. May have been longer, but what that essentially does is enables low level API access to some of the pieces of snowflake that you couldn't before, and I think what we're going to see out of that is applications coming out of that, built by partners, maybe some built by Snowflake, that are going to be the equivalent of lambda's on aws and some of these are going to make things easier, right, being able to go out to the data marketplace and get an ML model that's going to help you predict customer turn for clients that can't build these things in the house, that's gonna be a huge help to just start to play around with M L as opposed to going out and trying to figure out which tools do I need to buy? How do I build this model? What's going to be successful? It lets some experiment a little bit natively without a huge price tag and risk. All Right, last question for you. Gonna throw you a curveball, because I like to do that. What was one of the coolest implementations of snowflake that one of the team members had to build and what was the result of that? So I think one of...

...the cooler ones we've done was almost a data exploration exercise. There was a musical band that was branching out beyond just music that we were working with and, you know, they were starting to build out their brand and everything beyond just, you know, touring and selling albums, and it was cool to get access to all of this different data, right, and you know, merchandising data, and you know what they essentially asked for help with was can you take our data and help us figure out what we should care about, and being able to just throw that into snowfl really quickly and start exploring and do this over like a six day week period and then come back to them and show them exactly, you know, it's not productions data, it's not live data, but it's their real data and be able to show them here's what we found, here's the insights that we uncovered and here's where you should focus. Right like you have these core fans and this segment right here who are, you know, really big into, you know, one of the lines business that they were considering exploring and us being able to do that. Thank show that to them and something, you know, that's just not a big egg cell file. I thought that was a pretty cool way to use snowflake, as you know, almost like a prototyping exploration tool more than anything else. Well, Fred, I want to thank you for joining me as always today, as we covered how snowflake empowers you to unleash your business. Join US again next time on cloud crunch, where we will delve into the topics of all things cloud. Appreciate you joining us today. Fred, as always. Thank you very much. Thank you. You've been listening to cloud crunch with Michael Elliott and Fred Bliss. For more information, check out the blog second watch dot com. Forward Slash, cloud dash blog or reach out to second watch on twitter and Linkedin.

In-Stream Audio Search

NEW

Search across all episodes within this podcast

Episodes (43)