- Kevin Feasel
- Tracy Boggiano
Notes: Questions and Topics
Azure and Lego Pieces
I got things going with a rant about how things in Azure feel like Lego pieces which don’t quite fit together. The first five hours of an Azure project generally involves me enjoying how well all of these services fit together. Then, you find a spot where things don’t quite work and then you spend 15 hours trying to bludgeon the thing into submission.
My example was around using Azure Synapse Analytics. I used a Spark pool to train a model, and the notebook worked great, saving my model to Data Lake Storage Gen2 and everything.
Then, I wanted to deploy the model using Azure ML. Now, Azure ML lets you deploy PySpark models, so that’s no problem. The problem was, the model is in HDFS (really Data Lake but let’s not split hairs too much given that Blob Storage is an implementation of WebHDFS) but Azure ML won’t let you use Data Lake Storage for the model location—it needs to be local to the machine. Which means local to the driver. Which is the opposite of what you want with Apache Spark.
The end result was that I could get the pieces to fit, but it involved a bit too much welding, filing, and sanding for my taste.
Dealing with a Production Outage
Tracy tells a story of teamwork during an outage. I’d link to it, but unfortunately she’s dealing with a blog outage.
How is the DBA Role Evolving?
Mala couldn’t make it tonight but she did give us a great topic: how is the DBA role evolving?
I took it three ways, noting that this is focused on “production DBAs” rather than database developers:
- Is the DBA role evolving? My short answer is, I can argue either way. On the “yes” side, DBAs are expected to have more development skills than they had before. We want them to know about source control, Powershell, and DevOps. They also, in cloud environments, have fewer knobs to turn and lose ultimate control over the machine. But on the “no” side, the key set of DBA roles hasn’t really changed that much over the past several decades. Think about taking backups and testing restoration, ensuring high availability and/or disaster recovery, preventing or correcting corruption, tuning performance, and monitoring systems. The means have changed but the motivations and goals are the same.
- Can a DBA from 1985, 1995, or 2005 be teleported into today’s world and survive? Assume that there’s some time for learning about technological changes. Well, take a look at DBAs around you. Many of them got their starts in the 1990s, so the answer is probably “yes.” Now, the person from 1985 or 1995 would need to unlearn quite a bit and learn quite a bit, but I don’t think this is some crazy sci-fi future scenario where a DBA from then gets dropped in and has absolutely no clue how to be a DBA today.
- Will there be DBAs 10 years from now? 20 years from now? I think the answer is absolutely yes, because the things which define a DBA’s role don’t come about because DBAs wormed their way into organizations. They are critical for the business: we have data, data which the company finds valuable. As a result, we need to make sure that the data is available, that it won’t disappear tomorrow, that it won’t become corrupt tomorrow, and that business users will be able to get to it in a reasonable amount of time. There will be people who specialize in this administration of the bases of data as long as people at businesses care about data.