NetApp Data Science Toolkit
Transcript
all right have it all right go ahead mike awesome so hey keith uh thanks for joining us here yeah i i uh so i'm gonna go through the demo but then you know i don't have any sort of formal presentation or anything so feel free to jump in and you know like host seb said we're very interested in your your feedback so feel free to jump in with with feedback as well um unfortunately i oh it says host disabled screen uh let me allow screen sharing
yeah so keith just before before mike starts what we're trying what would be doing is you know we had the cloning snapshots for a very long time and we started thinking how can be that helpful for data science teams data engineering teams obviously the data science people that we hired we asked them you know what was the problems that you were going through they all mentioned that model versioning along with the data is one of the biggest problems that they have today think about it if
a data scientist today trains the model on a certain amount of data or certain types of data and after a year they want to retrain that model to go back to the results they had a year ago they are at the mercy of someone not tampering the data they are the mercy of that data scientist staying around so they know which data has been used usually we learned that most of them do it on a spreadsheet saying the training at this day was run on the
data existing in this share folder you can easily model the version that's not a problem github has that functionality most of the ml ops have that functionality but taking the data as part of the model version is one of the problems um oh my go-ahead yeah i think you have it over here yeah so it's really these these three slides that i showed on on thursday that kind of highlight the problems we were trying to solve right i mean it the first is uh creating you
know with the first is is the scenario where a data science you know data scientists or multiple data scientists are working off a gold source data set and you know running different experiments on that data and and so you know they don't want to modify the gold source but for each experiment they have to manipulate the data set in some way you know reformat it maybe into the right kind of feature vector to to fit the model architecture right you know run different normalization techniques and
so um what a lot of customers we talked to were doing were basically you know copy paste right and waiting hours while they copied these these data sets all over the place and there was kind of no tracking built in it was a very messy manual process and they were you know sitting around for 17 hours drinking a coffee right when you know we can help them clone a data set into you know in two seconds so let me understand the workflow so basically yeah the
if let's say that i have a bunch of baseball data right and this data is streaming let's say it's a bunch of streaming baseball data and i want to know uh the impact of of of uh attendance on a pitcher's era over a period of time uh i take a snapshot of that if i'm the data scientist i run that algorithm and query on a specific day i get a result i look at i look i start to munch on that data uh result i look
oh you know what my algo is off a little bit let me change the algo but now that data has changed so i don't know if it was the algo or if it's the data that fixes the problem or or was the problem so what they've done in the past is they take a snapshot of the data they copy a full work full data set to a share run it against that data and that's how they do their versioning versus just you know some intelligent way
of of doing versioning so so versioning is probably a misnomer for this this particular use case versioning more comes into place down here which which i'll get to in a second but basically it's it's not even as clean as that really what what we see a lot of a lot of these data science teams doing it's that they'll you know they'll have a a data set somewhere that they've kind of picked out and said like if we use that the picture era example they'll have based
on attendance they'll have kind of the the attendance data set right and they've decided you know we're going to train a model that that predicts pitcher era based on this attendance data set or something like that and and uh basically they'll the their goal is to train the most accurate model right so they've got one data set you know maybe you know attendance over the last 10 seasons compared to pitcher era or something like that that's their one data set and they'll say we need a
model that that fits this accurately that to predict picture era based on attendance and so they'll um basically what they're going to do is use it you know tweak different hyper parameters different aspects of the model architecture to train something like you know 10 20 30 different models to see which one is most accurate but each time they do that there's often kind of a a change required in the data set to support the the different model architecture so they'll they're you know it's not always
30 different times but if they're training 30 different models maybe they're copying their data set uh 10 different times right and each time they copy it it's it's often hours if it's a massive data set and so we said basically we can apply our cloning volume cloning technology to this and you know if you want to perform a another experiment to try and produce a more accurate model you know you don't have to wait hours to create a copy of your data set you can in
two seconds have a copy of your data set and get on with your experiment and if if you manipulate that data set you're not affecting your gold source at all so you could you know very quickly clone your data set train a model and maybe it doesn't look good throw it out you know clone it again train another model and and you've still got your your gold source there so it it does away with that problem of of kind of having to wait forever to make
copies and then having copies all over the place where where versioning comes into play is when uh and this is more so applicable to you know it at least we've found in talking to our customers to industries like financial services and healthcare right where if they're deploying a model to production their compliance department's not letting them put that model in production unless they have unless they can say they can fully explain the model and part of that is they need to be able to say this
is the exact data set we use to train this model right for sox compliance they need in in financial services at least they need full traceability there and so what they're kind of doing to support that is that kind of comes into play after the experimentation process where they've said okay this is my most accurate model this is the one i want to deploy into my production application that's let's say you know approving and denying customers for credit cards before they're going to be able to
deploy that they need to implement some sort of traceability that's like all right here's our deployed model here's the exact data set we use to train it we've got full built-in kind of immutable traceability there and and you know we heard that challenge and said hey volume snapshots they're a perfect fit for that right they have a long and proven history of being used in fortune 500 companies public sector dod right to near instantaneously version data sets they're a perfect fit for this and and so
you know a data scientist can take when they're ready to deploy a model to production they can take a snapshot of their data set uh and and just kind of save that snapshot id and a tracking tool you know it in their their model repository or model tracking tool as an attribute of the model and it says you know here's the here's the snapshot of the data set that's tied to this model i can go look at that snapshot and that's the exact data set i
used to to train this model does does that help you're on mute keith yeah so there's the two different use cases there's the switches which is you know let me just get a copy of the data so i can uh have multiple people working both multiple uh models against and then there's diverge versioning which is more applicable to the example that i gave which is you know what i want to use the same data set and a and the underlying in a in an underlying let's
just call it streaming data and data that constantly changes but i want a snapshot of that data and i want to be able to run different models across that data so it gives me in theory the ultimate flexibility if i have a team that's doing a bunch of different things against a set of data underlying streaming data then that team can either work multiple models against that data they can work uh different uh versions of the data or they can do a combination of different versions
of models in different versions of the data but they can intelligently kind of organize that that workflow exactly and and the traceability part is is kind of key too it really helps them efficiently implement that data set to model traceability for the the production models that's something that has really resonated with with a lot of our customers so are people building any workflows with like get to uh do that is or is this is this is this a big git community because i think i run
into this problem in hpc in general in which hbc seems like the perfect environment for containers and people but the community isn't into that uh not there is growing but they're not is not part of that culture is get part of the culture for data scientists in this workflow it is yeah so they're git is part of the culture but then they're it's it's not always get yeah there's kind of different levels of of maturity among data science teams you know there's the more advanced you
know kind of more super technical data scientists who come from you know places like google and and more of the the web scale type companies and and i mean they're building all kinds of custom workflows right and so they'll be more accustomed to using git and then there's the more kind of traditional enterprise data science teams and they're not necessarily accustomed to using something like git they're they're accustomed to you know they've got some experiment tracking tool like ml flow that they're logging all of their
experiments to um and say you know that also has a model repository that they're saving all of their models in but those experiment tracking tools don't really handle the data sets at all they're just kind of for you know the model that's what yeah so this allows for supporting both environments so if you're super into git you can create you know a super custom workflow but literally if you're just used to you know just working with nfs shares and and you know in the and in
the spreadsheet it was frankly in you know my my workflow is a spreadsheet at the minimum i can uh save the 17 hours and do two seconds and now i've just improved my existing workflow without changing the workflow itself exactly yeah and for the traceability piece i mean the idea is you could integrate it with whatever tool you're using right so you know you if you want to save your data set snapshot for traceability you know you can save that snapshot id and get yeah it
in your git repo you know tied to your model and you've got traceability there you can save it in your experiment tracking tool or like you said you could even save it in an excel spreadsheet yeah yeah and that integrity is just as strong as a as a hard copy because it's the it's the id and if i go back to uh my uh on type tap interface and that snap was deleted it's just like deleting the it was just like deleting anything else the traceability
is still there is the data may not exist but the traceability was still there and i can go back and look and say who deleted this the snap etc etc so it's just as robust as the previous correct actually even more robust than the previous solution because i have more headroom more capacity to hold more uh more samples more versions of the or more snaps of the data more data sets etc so it's as extensible as my environment can get exactly yeah exactly keith and most
of the time most of the time usually you know if you're training a model today you know the accuracy is like 0.0 difference percentages um and the third level down and then if you have 100 images you train them all year gone hundred images delete one image and that accuracy will change and usually data scientists will fall into this trap of whether the data was the same or i am doing the parameters wrong right so that that's where having the same data at least tells you
the data is right then let's look at the parameters that we changed a year ago or year after so it goes from being uh asking impossible questions i've talked about this in another realm but this is it's true on this case too which is the data scientists are asking themselves impossible questions that's impossible or near impossible to answer this gives them the ability to uh ask these data integrity questions that they can correct exactly yeah and and it gives them an answer when the the compliance
guy comes asking those questions right yeah i can see how to put a super beneficial in a regulated environment like a financial services or even uh drug discovery etc like how do i know that you're working off the same clinical data when the clinical data was updated two two months ago what was the updates now i got to go back and reconcile what was the updates from between when i ran the models originally versus when i'm running the models now and if there's a material difference
and something like drug discovery or financial services that's material that's a material question exactly yeah we've talked to several customers who were basically like you know they wouldn't let us go to production right we spent all this time working on this and then it it was just kind of wasted because we they compliance wouldn't let us go to production and they were looking at this as a potential solution to that so we want to jump over to the demo now yeah so basically what we did
next level keith is do we okay we we have this technology but we don't want the data scientists to learn about ontap they don't need to learn about ontap how can we expose all these features that we're talking about into a single point where they can use these features and that's why mike has been working to integrate it into the tools maybe mark you can show slide number six and and you can explain the layers that where the data scientists exist right yeah so so i
i was saying on thursday kind of you know during the the storage field day we uh you know we we had these solutions to the these problems right the cloning the snapshots but the data scientists the feedback we got from the data scientists and and the customers we were talking to is these are this is cool we really want to use this but it's not approachable right you're we have to learn your storage yeah yeah yeah i i've i've seen this problem really yeah when i
need to train we we deployed uh connection to internet2 and uh put a transfer node on it you know in the place where people can you know transfer transfer 100 gig of data overnight and the data scientists were just like you know what the data transfer tools i'm not familiar with it not approachable i'm just going to go to the ftp and chant trent and transfer the data over the internet because i know exactly yeah it's the same exact situation we were running into so it
was resonating but it wasn't really translating to you know customers actually taking advantage of of what we could offer and so um you know we had a lot of customers who kind of had purchased a ferrari right but it was parked in the garage right they weren't they weren't using any of these capabilities or they were driving 25 miles an hour right around the neighborhood um so so the yeah like hosep said that was kind of the goal of this data science toolkit was to to
wrap it wrap these capabilities in an interface that's going to be super simple uh and and easy for the for data scientists to use and also uh you know approachable like something familiar something that looks like the other tools that they're used to working with so it basically that it's a toolkit it's a python program that can function as either a cli utility you know for those more power user data scientists who are used to working from the terminal uh and and so it's just one
cli command to provision a volume clone of volume snapshot of volume right or for the the kind of less power user type data scientists they're they're most often working in jupiter notebooks right just coding python you know the python training jobs and so for you know and their nfs volume is is mounted just right there in their jupiter lab environment and so for them it's a it can function as an importable library of python functions so they you know like to clone a volume they just
import the clone volume function into their jupiter notebook right and then to clone a volume they just call the clone volume function so it's so it becomes just as simple as saying you want this process to take 17 hours or two seconds yeah if you do import this import this library and your uh you're good to go so because data scientists don't care about how much storage is used on the back vacuum that doesn't impact them they do care if they can save 17 hours exactly
yeah or if they can convince the compliance guy to let them go live right so the the yeah so now you know it's really been resonating in a way that our stuff wasn't before because now we can show them you know here it is here's how you do it and it looks familiar it's like oh you know that makes sense i work in jupiter notebooks i just you know import a function and call it i do that all the time right it looks like everything else
i do so it's the ideal is to go through each one of these environments that are super popular because this is the this is the when i see stuff like this this is the pushback and challenge that i give storage vendors specifically is changing the underlying habits of the dbas the data scientists etc sure cloning is great but i can't get an oracle dba not to do a zero level backup that's just how they that's how their minds work and they won't trust the storage array
to do it unless i made a zero uh level zero backup look if unless i made a clone look like a level zero backup then they're fine because they're using the same tools that they're already accustomed to using so if the idea is to go to as many of these tools that are common to the uh to the data scientists and make it as natural as possible yeah that's the goal so that's what we're trying to do here just to make it more approachable and and
really kind of help them consume these these capabilities this toolkit that we released it as open source right we're we're giving it away for free so it's really kind of the idea is you know not only for new customers who were looking at us you know for for their ai workloads but even for our existing customers who we already have you know who've already bought our storage and are already using it to support their ai workloads uh you know now they they can kind of take
advantage of more of these features so i like this so what you're basically saying is you're not this is not a new product and since this is saying look all the netapp goodness is perfect for uh ai and data science and all of that you're already using it today uh the we're just providing that extra value as to do the orchestration for you this this these bits and pieces that take some development work we'll do the development just use the online netapp storage to to do
it because in theory in practice if i'm doing this on-prem if i'm consuming uh netapp um cloud values and netapp services uh in azure this same the same process is available to me and anywhere i can get uh on tap um exactly yeah so that yeah this supports this toolkit is it's built on top of ontap rest apis right so it works with any flavor of ontap be that one of our on-prem appliances our software defined on tab select or the cloud volumes on tab so
which could be in any of the three clouds like you mentioned yep okay this is this was clever yeah so and jose mentioned the the use cases just real quick before we jump over to the demo yet the idea was and i we've kind of already touched on this but the idea was we wanted this to be something that data scientists could integrate into any existing tool that they're using right we wanted it to be simple enough to where they could just fit it into their
workflow with their existing tools is kind of a value add so it it could be standalone for smaller teams who don't have you know a lot of tooling or it you know they could fit it into their their existing mlops tool set be it a proprietary or even a homegrown ml ops toolset all right so let me jump over to the demo and hopefully this is gonna work unfortunately i'm in the process of rebuilding my lab environment and so i i'm gonna have to use the
youtube version of my demo and kind of skip through it my my simulators and everything in my my lab are down no we don't we don't have to go through the if there's a youtube to it just send me a link to how to take a look at it this was this other part of the conversation was way more valuable than actually seeing you know the demo i i can always reference that one yeah that that works for me i can i can send you uh
we've got two demos out there one is is clone is a cloning demo and and one is a snapshot demo so i can send you both of them and then there's actually a i'll send you this as well there's an interesting blog post that we put out that shows kind of uses ml flow as an example and shows how the the what i was talking about the snapshot id can be integrated into the experiment tracking tool to implement the traceability so i'll send that over to
you as well yeah i like this this is not disruptive to the existing workflow which is one of the things i just whenever i look at these storage enabling technologies from an i.t infrastructure perspective it makes perfect sense but what i've always run into where i've run into trouble with adoption is that there's something that's not approachable to unless i'm working with and i don't unless i'm working with a google or an advanced team that can build around apis data scientists and these groups and similar
groups just don't that's not that's not what they do so you know they learn the tools that they need and they and they use them yeah that's and that's exactly what we ran into when we first were pitching snapshots and clones to for for ai is that the adoption just wasn't there for that exact reason so that's that's the problem we're we're trying to solve and you know we we just released this little over a month ago so it's new but so far it seems to
be resonating yeah i think the the story makes sense i love when you guys get an update on just how it's been received i love to uh hear about how it's being received because that's kind of the the the money piece the that if you can crack that nut that's what we're trying keith no i think this is uh this is this is probably one of the more forward-thinking examples that i've seen of using existing storage because you don't need anything new other than the shim
in between the data scientists that make the the that makes the thing appealing to begin with right yeah all right keith any questions i hope this was useful time uh for you nope this was useful time i really it's been a while since i've taught the netapp in general so uh i really appreciate you guys taking up the time and and walking me through this uh i love it it was at my level and it was probably a little bit more uh consumable than the than
the field day theme which is kind of geared toward a broader audience yeah yeah thank you for your time
