Cloud Crunch
Cloud Crunch

Episode · 2 weeks ago

S4E6: Snowflake within the AWS Ecosystem


With only two episodes left in Season 4, today we will discuss "What key things do I need to know about Snowflake and AWS?" We are joined by our lead host and Director of Marketing Michael Elliott, and co-host Fred Bliss, CTO of all things data at 2nd Watch. Our special guest is Jason Mass, Executive Director of Data Insights for 2nd Watch.

Involve, Solve Evolve. Welcome to cloud Crunch, the podcast for any large enterprise planning on moving to or is in the midst of moving to the cloud, hosted by the cloud computing experts from Second Watch. Michael Elliott, Executive director of Marketing, and Fred Bliss, CTO of all Things Data at second Watch and now here are your hosts of cloud Crunch. Welcome back to a new season of cloud Crunch, and this season we're going to focus on a w S reinvent, the biggest cloud conference in the world. Our intent is to enable you, the viewer, opportunity to immerse yourself and how cloud has evolved since last year on topics like preparing and building the center of excellence, extracting data insights, managing a cloud native environment, and data center evacuation. Joining me today is Fred Bliss, CTO of Data Insights and Jason Moss, Executive director of the Data Insights Practice here at second Watch. Welcome to the cloud Crunch Fred and Jason, thank you, thank you for having us. Good to be here. So the focus of this video cast is snowflake within the AWS ecosystem, which, if there ever wasn't a perfect example. A friend of me is I don't know what is when we start to talk about Snowflake and a WS. So first question just kind of key this off, you know, for for folks in the aw S environment and for those folks thinking about snowflake, Now, what what do you need to know about snowflake on the AWS ecosystem? Yeah, I mean, I think the thing to...

...know is that a WS has a lot of native services that work really well with Snowflake. Snowflake is a enterprise data warehouse that UM, you can land data, you can transform it, and UM really really put it into a model that supports data and analytics. UM that's easily done with a w S in native services UM such as Amazon Glue. I think to to add onto that, UM, you know Snowflake when you think back to the in their history therefore they first launched as in a WS only solutions. So uh, it's been a long time. But for those UM who are just familiar with the current space, you know, you've you've got a flavor on Azure, you've got a flavor on Google Cloud. UM. Back then, when when the Azure announcement came out that Snowflake was coming on Azure, it was a big deal. It was. It was somewhere so you could go, UM and AWS and Snowflake. Uh they played nicely together, right and for a time Microsoft and Snowflake did as well. That's changing a little bit now and now with Google trying to capture more market share. UM, you've got that partnership as well. But as you alluded to, Michael, UM it is a little bit of competition, right because UM, the interesting thing about Snowflake is it runs on top of all these platforms, so they need AWS to exist, right. But at the same time, because of the way it works, and because the way you can replicate your data UM between clouds existing on Snowflake, that's got a UM it's an interesting Uh, it's an interesting take to be able to have your data living on Azure, AWS and g c p A at the same time. Right. So, like Jason said, there's a lot of services on AWS that coexist with it and interact with it, and it drives a ton of consumption for WS. But at the...

...same time you've got that UM that coexistence where uh moving your data or at least having copies of your data and other clouds is UM is a real thing. Now the one thing that you know, we hear a lot about and and you've alluded to Fred, is people coming and saying I want a specific tool, you know, I want Snowflake. And is that really the right question to be asking or is it really more about what is that business problem you're trying to solve? So can you expand a little bit upon that? Yeah, if if we are ever getting it, you know, if I'm if I'm having a discussion with UM, with a potential customer or an existing customer about about products, you know, should they use Snowflake, should they use big Queries, should they use something else? Um, we're having the wrong discussion, right because, UM, we don't know, we don't know necessarily what they need the uh uh. You know, as we start to dive into understanding what is the business problem we're trying to solve for now and what vision are we trying to build for as a whole? Right, UM, that can feedback into the products because if you make a product decision based on what you know today and UM, you know, just based on what your past experience was and maybe your maybe your use case system migrate from an old system to a new system, UM, you're missing out on the much bigger picture of what could UM. What were you not able to do in the past because of technology limitations that you can now do today because of current technology solutions. So I think you have to go down that path because your technology decisions will derive We'll drive everything because there is no perfect technology, there's no perfect product. It's all going to be a series of trade offs. I can agree more. I mean, really, we we want to start looking at the big picture first. We don't to start with the technology.

We never do UM. You know, really we're focused on solving business problems. So really understanding what the clients trying to do UM, what is what are they trying to use this platform for UM and what what are they trying to solve As far as UM understanding some of their outliers and UM really key performance indicators at at that organization. UM, and you should never start with the technology. Technology only it helps that enable There's a lot more UM. There's a lot more to building a data and analytics platform than just the technology. You've got to consider the people in process changes as well as really just identifying what's going to bring value to that organization. So really leading with the technology and picking a technology without understanding that first, UM is kind of a no no. So we try to avoid that. How often do you go into situations where they had picked a tool in the past and now that tool is dead or you know, what they want to achieve won't work on that tool. I mean, is that something that happens. I mean, I'm an old Cobalt programmer, so I know we don't go we don't program in Cobo anymore, but the same does that same op? Yeah? Absolutely, I mean I think think about like just even organizations that decided to stay on premise and and um really try to build a data and analytics environment. What they're running into is the same problem that we've had over and over in the in the you know, since I've been uh doing building these type of platforms, which is about twenty three years, is that you know, when you build on premise, you don't have that power to scale. I mean you do have it, um, but it takes a long time to get there. Where Snowflake, Uh, you know, being on Snowflake, you you can scale up anytime. UM. So I think it. You know, you're still seeing I'm seeing people replace sp Hannah and really because they they're they're...

...lacking in the ability to scale for their organization, so shifting and sorry, just to add on to that, I think, UM, you know, early in my career, UM, you know, I think we all learned through a lot of failures, right and UM as in this ecosystem, you see products coming in and out right and there's the latest flavor, whatever is hot. You know, you've got some uh, some tech companies that just jump on and latch onto whatever the latest cool thing is. UM as a company, we you know, for example, we we never did Hudoo. We never implemented it because we didn't see the value and I knew there was a complexity in it. So UM, I think you always have to be aware of, UM, what's the standing power of this technology? Is it going to be around as a company in two years? Is it going to get acquired? Um? Because the last you want to do is recommend to a customer. Yeah, I think you should use this as a core part of your technology foundation and then a year later it's integrated into a platform that you just moved off of. So we're talking about friend of these earlier. So Snowflake versus red shift, which which one do you use. I mean, how do you know which one is the proper tool for what you're doing? Are there advantages of one over the other? I think it's more Um, I mean, if you think about it from a technology standpoint, Uh, there's a couple different flavors of redshift, and I know a WS is working on new flavors of redshift all the time. But the fact of the matter is, Snowflake is a newer technology. It is a cloud native technology, whereas redshift is, Uh, it's closer. There is a severalist variation, but it's essentially a port of fork of post grass. Um, so it's still very much a traditional database. But that said, UM, if...'re an organization, especially large enterprise, and you just want to get something going so that you can show to business usures, uh, something that works. And I've got examples of clients doing this right now. UM. You don't necessarily have the time or the political capital to go sign an m s A with a third party vendor. You just need to get something going fast to prove the value of why what you're doing is important. And for that case, you know, red shift is a great way to start into it. And uh, you know We've got some clients that did it for exactly those reasons and they never outgur it, so there was no reason or will or UM I guess strong um motivation to move off of red Shift. It worked for what they needed, and it probably will work for what they need until it doesn't. Right. But the good thing is it's all sequel based and we're building it all in services that live with they need a bus. Um, it's just data, right, Yeah, I mean the bottom line it's it is a cloud enterprise data platform just like Snowflake and UM red Shift and Snowflake and Sinnapse and big Querry. Those are your enterprise data warehouses by vendor and really they they all will do the job. Again, it goes back to you know, what are you trying to accomplish? And you know, if you understand that, you can build really any solution with either of these platforms, with red Shift or Snowflake. So next question for you, how do Snowflake in aws best work together? I think they work best together? And that UM, I mean Snowflake wants to be a cloud within a cloud, right, so UM explain that what what what do you mean by a cloud within a cloud? Yeah, I mean if you look at some of their recent acquisitions like Streamlet for example, and some of their UM their new announcement with Unistore, which is essentially turned Snowflake into a transactional database,...

...meant more for applications. UM. The fact that you can now deploy applications Python applications. Uh. You know, you've got snow park, which opens up a p I S to some of the low level Snowflake capabilities. I think of it like you would AW s LAMB does. Right. Um, you can deploy code to Snowflake that isn't just sequel anymore. And so if you want to build your applications within Snowflake, which is sitting on a w S, you can absolutely do that, right, and now you can make that portable between the two. At the same time, by doing this, you're you're embedding yourself even further into the Snowflake ecosystem. UM. So you kind of got to pick where you want to be, right. You could deploy these applications in AWS, which then communicate with Snowflake as the back end database, or you can start to put everything into Snowflake UM, and at that point it's really more about Snowflake being the driver and the cloud within a cloud and less about AWS. But at the same time they have to coexist because you still need things like, uh, you know, identity and networking and all that stuff that goes into a cloud infrastructure projects. And I think to a degree you kind of got to get into the last question I have, which is, you know, what are some of those benefits of using Snowflake on a WS. So I think you've kind of got into that, but what are some of the other options that are out there if we're just talking to w S. I mean, there's there's a lot out there, right, and there's there's some emerging players out there. Whether they're going to have the um the lasting power, I don't know yet. But ClickHouse is a is a popular open source one. They just actually released a cloud sas version. UM. They've been around for a while. I think they spun out of Yanda, if I remember, which was basically the Russian search engine.

Uh. There's there's also on a w S. There's a lot of different ways you can do. You can use Athena, which is essentially Presto under the hood. Um, and then there's all the RDS databases. So at the end of the day, it's really just about what's your use case of what are you trying to solve for? Um? Well, how big is your vision and what are you trying to do? And again it's a lot of trade offs. Which one is going to help you best is going to depend on what your individual um your individual use case and long term vision is. Well. Excellent. Well, I want to thank you Fred Jason for joining us today to discuss you know, snowflake within the AWS ecosystems. Um Jason, any final words of advice for attendees of Reinvent. Yeah, I mean, I think I think kind of the whole message of this is, you know, don't start with the technology, you know, think about the overall data strategy for your organization. UM. You know, start with a data strategy, right you. You wouldn't build a house without an architecture diagram without a plan. Um, it's the same for building a Dana analytics platform to support your organization. Start with that data strategy. Understand from the business what's what's a priority for them, What's what's what are those key performance indicators that they're looking to measure and if if you do that that that's usually a successful implementation that can happen because you had that plan. So start with plan, don't don't start with the technology. xCE well, I want to thank our audience for listening to the show. This video cast is intended to add value to any large enterprise that is planning on moving to or is currently focused on leveraging the value of the cloud. Send your comments or suggestions to cloud Crunch at second watch dot com. Thank you you've been listening to cloud...

Crunch with Michael Elliott and Fred Bliss. For more information, check out the blog second watch dot com, forward Slash Cloud dash blog, or reach out to second watch on Twitter and LinkedIn.

In-Stream Audio Search


Search across all episodes within this podcast

Episodes (43)