Conversation with Dirk-Jan Kubeflow (vs cloud native solutions like SageMaker) Dirk-Jan Verdoorn - Data Scientist at Dept Agency Kubeflow. (From the website:) The Machine Learning Toolkit for Kubernetes. The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Our goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML to diverse infrastructures. Anywhere you are running Kubernetes, you should be able to run Kubeflow. TensorFlow Extended (TFX). If using TensorFlow with Kubeflow, combine with TFX for maximum power. (From the website:) TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines. When you're ready to move your models from research to production, use TFX to create and manage a production pipeline. Alternatives: Airflow MLflow 

Machine Learning Guide

MLA 020 Kubeflow

Chatting with co-workers about the role of DevOps in a machine learning engineer's life Expert coworkers at Dept Matt Merrill - Principal Software Developer Jirawat Uttayaya - DevOps Lead The Ship It Podcast (where Matt features often) Devops tools Terraform Ansible Pictures (funny and serious) Which AWS container service should I use? A visual guide on troubleshooting Kubernetes deployments Public Cloud Services Comparison Killed by Google aCloudGuru AWS curriculum 

MLA 019 DevOps

(Optional episode) just showcasing a cool application using machine learning Dept uses Descript for some of their podcasting. I'm using it like a maniac, I think they're surprised at how into it I am. Check out the transcript & see how it performed. Descript The Ship It Podcast How to ship software, from the front lines. We talk with software developers about their craft, developer tools, developer productivity and what makes software development awesome. Hosted by your friends at Rocket Insights. AKA shipit.io Brandbeats Podcast by BASIC An agency podcast with views on design, technology, art, and culture. Explore the new microsite at www.brandbeats.basicagency.com 

MLA 018 Descript

Show notes: ocdevel.com/mlg/mla-17 Developing on AWS first (SageMaker or other) Consider developing against AWS as your local development environment, rather than only your cloud deployment environment. Solutions: Stick to AWS Cloud IDEs (Lambda, SageMaker Studio, Cloud9 Connect to deployed infrastructure via Client VPN Terraform example YouTube tutorial Creating the keys LocalStack Infrastructure as Code Terraform CDK Serverless 

MLA 017 AWS Local Development

Part 2 of deploying your ML models to the cloud with SageMaker (MLOps) MLOps is deploying your ML models to the cloud. See MadeWithML for an overview of tooling (also generally a great ML educational run-down.) SageMaker Jumpstart Deploy Pipelines Monitor Kubernetes Neo 

MLA 016 SageMaker 2

Show notes Part 1 of deploying your ML models to the cloud with SageMaker (MLOps) MLOps is deploying your ML models to the cloud. See MadeWithML for an overview of tooling (also generally a great ML educational run-down.) SageMaker DataWrangler Feature Store Ground Truth Clarify Studio AutoPilot Debugger Distributed Training And I forgot to mention JumpStart, I'll mention next time.

MLA 015 SageMaker 1

Server-side ML. Training & hosting for inference, with a goal towards serverless. AWS SageMaker, Batch, Lambda, EFS, Cortex.dev

MLA 014 Machine Learning Server

MLA 013 Customer Facing Tech Stack

Use Docker for env setup on localhost & cloud deployment, instead of pyenv / Anaconda. I recommend Windows for your desktop.

MLA 012 Docker

Show notes at ocdevel.com/mlg/32. L1/L2 norm, Manhattan, Euclidean, cosine distances, dot product Normed distances link A norm is a function that assigns a strictly positive length to each vector in a vector space. link Minkowski is generalized. p_root(sum(xi-yi)^p). "p" = ? (1, 2, ..) for below. L1: Manhattan/city-block/taxicab. abs(x2-x1)+abs(y2-y1). Grid-like distance (triangle legs). Preferred for high-dim space. L2: Euclidean. sqrt((x2-x1)^2+(y2-y1)^2. sqrt(dot-product). Straight-line distance; min distance (Pythagorean triangle edge) Others: Mahalanobis, Chebyshev (p=inf), etc Dot product A type of inner product. Outer-product: lies outside the involved planes. Inner-product: dot product lies inside the planes/axes involved link. Dot product: inner product on a finite dimensional Euclidean space link Cosine (normalized dot) 

MLG 032 Cartesian Similarity Metrics

Social media <a href="https://gnothiai.com/" rel= "nofollow">Gnothi</a> and email me a screenshot/link for 3-month access to <a href= "https://www.patreon.com/machinelearningguide" rel= "nofollow">Machine Learning Applied</a>; commit code to the <a href="https://github.com/lefnire/gnothi">Github repository</a> for life-access. Normed distances <a href= "https://medium.com/@kunal_gohrani/different-types-of-distance-metrics-used-in-machine-learning-e9928c5e26c7" rel="nofollow">link</a> <ul> <li>A norm is a function that assigns a strictly positive length to each vector in a vector space. <a href= "https://towardsdatascience.com/how-to-measure-distances-in-machine-learning-13a396aa34ce" rel="nofollow">link</a></li> <li>Minkowski is generalized. <code>p_root(sum(xi-yi)^p)</code>. "p" = ? (1, 2, ..) for below.</li> <li>L1: Manhattan/city-block/taxicab. <code>abs(x2-x1)+abs(y2-y1)</code>. Grid-like distance (triangle legs). Preferred for high-dim space.</li> <li>L2: Euclidean. <code>sqrt((x2-x1)^2+(y2-y1)^2</code>. <code>sqrt(dot-product)</code>. Straight-line distance; min distance (Pythagorean triangle edge)</li> <li>Others: Mahalanobis, Chebyshev (p=inf), etc</li> </ul> Dot product <ul> <li>A type of inner product. Outer-product: lies outside the involved planes. Inner-product: dot product lies inside the planes/axes involved <a href= "https://www.quora.com/Whats-the-difference-between-inner-product-dot-product-and-scalar-product/answer/Urgunoon-Saleem" rel="nofollow">link</a>. Dot product: inner product on a finite dimensional Euclidean space <a href= "https://www.quora.com/Whats-the-difference-between-inner-product-dot-product-and-scalar-product/answer/Richard-Morris-34" rel="nofollow">link</a></li> </ul> Cosine (normalized dot)

Social media Gnothi and email me a screenshot/link for 3-month access to Machine Learning Applied; commit code to the Github repository for life-access. Normed distances link A norm is a function that assigns a strictly positive length to each vector in a vector space. link Minkowski is generalized. p_root(sum(xi-yi)^p). "p" = ? (1, 2, ..) for below. L1: Manhattan/city-block/taxicab. abs(x2-x1)+abs(y2-y1). Grid-like distance (triangle legs). Preferred for high-dim space. L2: Euclidean. sqrt((x2-x1)^2+(y2-y1)^2. sqrt(dot-product). Straight-line distance; min distance (Pythagorean triangle edge) Others: Mahalanobis, Chebyshev (p=inf), etc Dot product A type of inner product. Outer-product: lies outside the involved planes. Inner-product: dot product lies inside the planes/axes involved link. Dot product: inner product on a finite dimensional Euclidean space link Cosine (normalized dot)

032 Cartesian Similarity Metrics

Kmeans (sklearn vs FAISS), finding n_clusters via inertia/silhouette, Agglomorative, DBSCAN/HDBSCAN

MLA 011 Practical Clustering

NLTK: swiss army knife. Gensim: LDA topic modeling, n-grams. spaCy: linguistics. transformers: high-level business NLP tasks.

MLA 010 NLP packages: transformers, spaCy, Gensim, NLTK

matplotlib, Seaborn, Bokeh, D3, Tableau, Power BI, QlikView, Excel

MLA 009 Charting tools

EDA + charting. DataFrame info/describe, imputing strategies. Useful charts like histograms and correlation matrices.

MLA 008 Exploratory Data Analysis

Machine learning audio course, teaching the fundamentals of machine learning and artificial intelligence. It covers intuition, models (shallow and deep), math, languages, frameworks, etc. Where your other ML resources provide the trees, I provide the forest. Consider MLG your syllabus, with highly-curated resources for each episode's details at ocdevel.com. Audio is a great supplement during exercise, commute, chores, etc.