Deep Learning for Tabular Data: A Bag of Tricks | ODSC 2020

Jason McGhee, Senior Machine Learning Engineer at DataRobot, has been spending time applying deep learning and neural networks to tabular data. Although the deep learning technique can prove challenging, his research supports how valuable it is when using tabular datasets. In this video (adapted from his presentation at ODSC Boston 2020), Jason shares some important techniques for implementing deep learning when learning heterogenous tabular data. Learn more about Jason’s findings and ask him questions at his DataRobot Community post: community.datarobot.com/t5/ai...
Table of Contents
Motivation: 0:15
Impute missing values: 1:37
Prepare categoricals, text, and numerics: 2:49, 3:10, 3:31
Properly validate: 3:54
Establish a benchmark: 5:24
Start with a low capacity network: 6:10
Determine output activation and loss function for classification and regression: 7:17, 8:26
Determine hidden activation: 9:46
Choose batch size: 10:57
Build learning rate schedule: 12:02
Determine number of epochs: 14:35
Track and interpret regression predictions: 15:30
Track metric and/or loss: 16:09
Track and interpret classification predictions: 16:45
Benchmark the network: 17:11
Dealing with discontinuities: 18:16
Tuning the network: 19:31
Handing overfitting vs. underfitting: 20:41
All tricks in one place: 21:35
Music for this video: www.bensound.com.
Stay connected with DataRobot!
Blog: blog.datarobot.com/
Community: community.datarobot.com/
Twitter: / datarobot
LinkedIn: h / datarobot
Facebook: / datarobotinc
Instagram: / datarobot

[@markryan2475]

This is awesome - so glad to see some serious, methodical work on this.

[@michaeljuhasz1162]

Thank you for this. As material’s science researcher dabbling in applying ML techniques to my datasets, this is great.

[@briantroy9403]

I’ve also noticed a lack of emphasis of tabular data with respect to NN’s. This is a great presentation and very informative. Thanks for putting it together.

[@alirezaamani2027]

very interesting point on the suggested loss functions based on the distribution of the target variable. Learned a lot. Thank u

[@ZachMeador]

Great use of Grant Sanderson's graphics library

[@parmarsuraj99]

This helped me a lot. Thank you 🙏😍

[@sayedathar2507]

Thanks for Sharing this are golden advices !

[@rupjitchakraborty8012]

Amazing amazing video, I learnt so much. Thank you

[@tahamagdy4932]

Great, but the music is very loud.

[@venk8t]

The background music is a distraction and hard to listen .

[@comunedipadova1790]

incredible, how to ruin a video

[@hungdoan9148]

Amazing video!

[@vibhatha]

Nicely done 👌

[@marekglowacki2607]

3Blue1Brown style :)

[@robimalco]

This is high quality content

[@MiguelRaggi]

By the way, one hot encoding and making an embedding are the same thing, except embedding is faster. What do you do with your one hot encoding? you multiply by a matrix. What happens when you multiply a vector with 0's and only one 1 by a matrix? That's right, you basically choose a column of the matrix. And that column is the embedding.

[@Houshalter]

My thoughts too. The only exception I could think of is if the software he used treated embedding differently somehow. Maybe regularization or dropout wasn't applied to the embeddings. Maybe normalization on the one hot columns had some beneficial effect.

[@idabagusdiazagasatya9900]

This is gold, keep it up.

[@parmarsuraj99]

Yes it is

[@mohamedesdairi3044]

you helped me A LOT, amazing content and prefect presentation, keep it up

[@DataRobot]

We are! Reply with topics you'd like to see covered.

[@mohamedesdairi3044]

@@DataRobot personally, I think topics like solving challenges in fitting neural networks to a small dataset, debugging why you're NN can't perform well on a specific task, and active learning are subject that don't get the attention they deserve just like "Deep Learning for Tabular Data" which you did a fantastic job in covering in this video, so thanks again

[@DataRobot]

​@@mohamedesdairi3044 Hey would you please ask your question in the DataRobot Community? You'll find a lot of community members who are interested in posting about DataRobot, machine learning, data science subjects. community.datarobot.com/t5/platform/bd-p/platform-discussions-1

[@TheOraware]

how to spot random subset of data from a given set of data?

[@mehdiozel517]

Thanks, really cool video! But I have a question. You said "set the batch size 1% of dataset". Is these informations provided for deep learning on tabular datasets or other types of data too?

[@brianray9715]

I dig the background music

[@anon44492]

Great work bro!

[@username42]

what about the 1d sensory data collected from physical and chemical instruments ? i know we can still treat them as tabular data but what about when we have thousands of variables and hundreds of samples only and the variables are not single identity but they are sort of grouping features , how to treat the data analysis ?

[@DataRobot]

​Hey would you please ask your question in the DataRobot Community? You'll find a lot of community members who are interested in posting about DataRobot, machine learning, data science subjects. community.datarobot.com/t5/platform/bd-p/platform-discussions-1

[@jivan476]

Nice vid, but tbh this sounds like a crazy amount of work for something that will only ever tangentially approach boosted trees performance on most tabular datasets

[@ThanhNguyen-rz4tf]

What is the paper you mentioned ?

[@morowenka9718]

amazing video 🥵

[@swarajshinde3950]