A new way to get started with Deep Reinforcement Learning

Every time I read an OpenAI blog post I'm inspired.

I'm inspired by the artwork, I'm inspired by the quality of work, I'm inspired by the communication skills.

They work on hard problems. Get great results. And show them off to the world in a way easily accessible to the world.

So when their post came out about Spinning Up, a free resource for learning Deep Reinforcement Learning (RL), I wasn't surprised. Not surprised by the fact I loved the artwork, by simple and effective communication in the blog post and the world class quality of the learning materials.

Seriously. This is something I'd put on my wall (stay tuned, this might happen).

 I’d love whoever designs OpenAI’s graphics to come and paint my room. Source:  OpenAI Blog .

I’d love whoever designs OpenAI’s graphics to come and paint my room. Source: OpenAI Blog.

More to the point, if you're interested in Deep RL, you should leave here and check out Spinning Up.

If you're not interested, it's worth knowing a little about it anyway.

Deep RL involves the training of an agent in an environment to learn something about the world.

This is best explained with an example.

Deep RL was the technique DeepMind used to build Alpha Zero. The best Go player in the world.

In Alpha Zero's case, it (the agent) learned to play Go by continually playing itself. It got to a superhuman level exceptionally fast because it was able to play many simulated games of Go (the environment) against itself at once.

The deep part comes from using many versions of itself to learn. So really, there were multiple agents (different versions of Alpha Zero, converging into one) and multiple environments (many different games of Go).

How about another example?

Let's say you're a doctor who wants to try a new treatment on your patients. But you're hesitant because you're not sure how it will go.

So you decide to wait for more trials to take place.

But trials are lengthy, expensive and potentially harmful to those involved.

What if you could simulate a treatment (the agent) and test it on many different simulated patients with characteristics similar to yours (the environment) and see what the outcomes were?

Using the knowledge you gather, you could potentially find an ideal treatment for each individual patient.

This example simplifies the process dramatically but the principles of Deep RL are there.

The beautiful thing is, Deep RL could potentially be harnessed for any problem where many different scenarios need to be accounted for.

Say you wanted to improve traffic lights to find a more ideal schedule, Deep RL could be used to test many different car arrivals (the environment) and many different light configurations (the agents) with the goal of maximising efficiency.

The scenarios are endless.

I know there's a way to somehow use Deep RL for health. But I don't know how (yet).

To help figure this out, I'm incorporating the Spinning Up materials into my curriculum for 2019. I can't wait.

And of course, I'll be sharing what I learn.

If you're interested in getting started with Deep RL, there are a few resources you might want to check out:

There's also a workshop OpenAI are hosting on February 2 at their San Francisco HQ for those who have tinkered with machine learning and are wanting to learn more about Deep RL. I applied and you can too (applications close December 8).

How to Learn Deep Learning in 2 Months

‘Hi, I’m a machine learning engineer from Google.’

‘Hello everyone, I’m a software engineering at Intuit.’

‘Good morning, my name is Sandy, I’m a freelance data scientist.’

Ugh.

I was scared to post my introduction.

What was I going to say?

‘Hi, I’m Daniel, I quit the first programming course I signed up to, so now I’m here to try again.’

I probably could’ve said that. That’s actually not too bad. At least it was the truth.

It was a week before the Udacity Deep Learning Nanodegree began. Everyone was introducing themselves in the Slack channel.

I was the odd one out. It seemed everyone had already some experience with machine learning or data science and here I was writing my first line of Python 3-weeks prior.

The course went on. I graduated 4-months later.

Then I went on and did Udacity’s AI Nanodegree as part of my online AI Masters Degree.

And Andrew Ng’s deeplearning.ai course on Coursera.

Then I got hired as a machine learning engineer.

All in the space of a year.

But 2 months? Woah.

That’s some ambition. I love it.

Let’s be clear though, the field of deep learning and all the other encompassing crafts is huge. You’ve seen that photo of the iceberg right? The one where most of it is under water and only the top 10% of it sticks out of the water. Yeah, that’s like deep learning.

The art of building deep learning neural networks can be picked up in a couple of months by someone with a little programming experience.


So how would you do it?

Done some Python work before? Go ahead with these steps. If not, learn some Python first and come back.

These two books will help you go from a deep learning rookie to a deep learning practitioner.

‘But Daniel, these books cost money?’

You’re right. But the education is worth it. Especially since $100 worth of books, if used correctly, could give you provide you the skills for a $100,000+ role. Or better still, provide you with the skills to build something incredible.

If you can’t afford these books, I’d recommend Siraj Raval’s or Sentdex’s YouTube channel.


‘I’ve got a little longer than two months.’

Putting together a deep learning model is often one of the last pieces of the puzzle.

Even the best deep learning models won’t help anyone if there’s no data for them to learn on.

Deep learning is the combination of statistics, programming, math and engineering. Maybe a few more things.

If you’ve got longer than two months, I’d still recommend the two books above. As well as any of the courses I’ve mentioned.

But the key thing is to try out what you’ve learned.

Find a problem which interests you and see if you can apply deep learning to it.

‘Yeah, people always say that, find something that interests you, but I’m not interested in anything…’

Not interested in anything?

Then what pisses you off?

Maybe you’re pissed off at the amount of homeless in your city. How can you use deep learning to find more people a home?

Or perhaps you don’t like the state of the healthcare system, does deep learning have a place in building a healthier world?

A major part of being a good deep learning engineer is to seek out new ways of looking at and solving existing problems.


How long per day?

I heard an interview once with a world chess champion.

‘How much do you play per day?’

‘Never more than 3-hours per day.’

‘Really?’

‘Yeah, but during those 3-hours, all I’m doing is playing chess.’

3-hours per day? That’s it?

I had to try it. So I did.

I set a timer. One hour, 5-minute break, another hour, 15-minute break, then another hour.

It took me all three hours to get through one function.

The next day I finished the whole assignment in the same timeframe.

I still do this kind of study every day.

Sometimes you may be able to do more. Other days less. But the main thing is to focus on one thing at a time.

Especially if you want to learn deep learning in two months. Block all the other crap out and learn deep learning. I have to remind myself of this all the time.

Never half ass two things, whole ass one thing. You’ll be surprised what you’re capable of.

You might even end up running down your street after finishing your final coding project.

 Study hard. Celebrate hard. Screenshot from Learning Intelligence 34:  Finishing the Udacity Artificial Intelligence Nanodegree

Study hard. Celebrate hard. Screenshot from Learning Intelligence 34: Finishing the Udacity Artificial Intelligence Nanodegree



Source: https://qr.ae/TUhEPQ

How much math do you need to start learning machine learning or deep learning?

‘I’m studying artificial intelligence.’

‘What?’ my Uber passenger asked, ‘is that like aliens?’

‘It’s more like a combination of math, statistics and programming.’

‘You’ve lost me.’

I didn’t really want to go back to university so I made my own Artificial Intelligence Masters Degree using online courses. I drove Uber on the weekends for 9-months to pay for my studies.

Now 7-months into being a machine learning engineer, I’m still working through it, along with plenty of other online learning resources.

My first line of Python was 3 weeks before starting Udacity’s Deep Learning Nanodegree.

I’d signed up and paid the course fees but I was scared of failing so I emailed Udacity support asking if I could withdraw my payment and enrol at a later date.

Screen Shot 2018-11-23 at 9.15.03 pm.png

I didn’t end up withdrawing my payment.

For 3/4 of the projects, I needed an extension. But I still managed to graduate on time.

The main challenge for me was the programming, not the math.

But to learn the math knowledge I did need, I used Khan Academy.

Especially:

I was learning on the fly. Every time a concept came up I didn’t know about, I’d spend the next couple of hours learning whatever I could. Most of the time, I bounced between Stack Overflow, Khan Academy and various machine learning blogs.

How much math do you need?

It depends on your goals.

If you want to start implementing deep learning models and achieving some incredible results, I’d argue programming and statistics knowledge are more important than pure math.

However, if you want to pursue deep learning or machine learning research, say, in the form of a PhD, you’ll want a strong math foundation.

The math topics I listed above take years, even decades to fully comprehend. But thanks to deep learning and machine learning frameworks such as Keras and TensorFlow, you can start replicating state of the art deep learning results with as little as a few weeks of Python experience.

The thing with math is, it’s never going away. Math is the language of nature. It’s hard to go wrong brushing up on your math skills.

But if the math is holding you back from jumping in and trying machine learning or deep learning, don’t let it.

A fisherman doesn’t learn how to catch every single fish before he goes fishing. He practices one fish at a time.

The same goes for learning. Rather than trying to learning everything before you start, learn by doing, learn what you need, when you need.

Source: https://qr.ae/TUhEPQ

The best $250 I've ever spent

‘Do you have a Queensland driver's licence?'

'Yes,' I replied.

'Well, you should know you're not supposed to stop at bus stops,' he continued, 'you'll be getting a fine in a couple of days.'

'Okay, thank you,' I wound up the window and drove off.

'That sucks man, that guy was a dick,' my passengers felt sorry for me.

'It's alright, these things happen, how was your night?'

We chatted for the next 30-minutes or so.

'Do you think I should see go on another date with her?' He showed me a photo of his latest Tinder match.

'She's not my type,' I said, 'not enough hair for me.'

The girl in the photo had a shaved head.

'Yeah you're right, it's different, but I think I like it,' he got out of the car, 'thanks, mate!'

I ended the trip. We had a good conversation so I rated them 5-stars.

There was no one on the highway. Pure open road. Driving at 3 am has its perks. Plenty of time to think.

'I'm not driving Uber anymore,' I thought to myself.

I parked the car, went inside and into bed.

More thinking.

'I won't have an income for a while, how will I pay for things?'

'I'll figure it out.'

The fine came a couple days later. It was $250. 6-hours driving and I made -$37.


The next day I peeled myself out of bed. Waking up was always an effort after going to sleep at 4 am.

My schedule for the past year had been, study Monday-Friday and drive Uber Friday night, Saturday night and Sunday.

It was Saturday. I was supposed to be driving Uber later that night.

I remembered what I said to myself before going to bed. All of a sudden it seemed a lot harder.

Driving Uber was the easy option. I loved meeting new people every trip. But it wasn't for me anymore.

'Well Daniel, better do something about it,' is probably what someone would've said if I told them my dilemma.

'You're right,' is what I would've replied.


I went to the cafe and posted on LinkedIn.

'Hey data science community, I'm looking to get involved in the field and would love your recommendations on who I should follow.'

The past year, I'd been studying my own Artificial Intelligence Masters Degree. It wasn't an official Masters Degree but I gave it that name for the placebo effect.

The post got lots of replies. People were commenting and tagging other people.

I spent the next couple of hours messaging every single one of them.

Ben_Taylor_Screen_Shot.png

Ben told me to become an expert in my field.

'How could I be an expert, I've only been studying for a year?'

I figured instead claiming to be an expert, I could share what I've been up to.

So I did.


A couple of days later, Ashlee sent me a message.

'Hey, Daniel, bit of a random thought but would you be interested in an internship opportunity?'

We talked some more.

'Mike will catch up with you to find out more about what you've done and what you are interested in so they can position you,' she said.

'Okay, where should I meet him?' I asked.

The next week Mike and I met at Brew cafe.

'How did you find the float tank?'

I read an article on his LinkedIn where he told a story about going into a sensory deprivation tank. I'd been in one too.

'It was spectacular, you fade away into nothing,' Mike said.

I agreed.

If you haven't yet, go in one.

'What are you looking for?' Mike asked.

I didn't really know. I'd been studying online but hadn't had any experience putting what I had been learning into practice.

'Experience,' I replied, 'online courses are great but I'd like to contribute to something tangible.'

'The crossover of technology and health is what interests me most,' I continued.

'If you get in front of someone, you'll get a job,' Mike went on, 'I think you should meet my friend Cam.'

I'd been thinking of moving to the US to get a role in tech.

'What do you think about the US?' I asked Mike.

'It's the holy grail for this stuff, but Brisbane is good too, I think it would be good if you got some experience before heading over.'

'Good idea.'

Mike paid for the coffees.

'I'm off to an interview at Code Camp, thank you for this morning.'

'No problems, talk soon.'

Mike and I shook hands.

I walked down the street and up to the Code Camp offices. Charlie offered me a job as a teaching assistant for school kids learning to code on their holidays. I said yes. But it wasn't anything long term.


Two days later, I booked a one-way ticket to the US.

I had no plans but to get on the ground and meet people. Then maybe someone would like me enough to offer me a job.

My thought pattern was, 'If you want to learn French, go to France.'

I wanted to learn the latest in technology innovations so I was going to Silicon Valley.

That was the good reason. The reason I told everyone why I booked it.

But the real reason was I didn't think I had it in me to get a job in my hometown. So I decided to run away. To the other side of the world.

It was 99-days until I'd fly out.

Then Mike messaged me introducing me to Cam.

'How about we meet up on Monday?' Cam asked.

I had no commitments set in stone since I was creating my own study schedule.

'Sure,' I replied.

Our chat went much the same as with Mike.

Health, technology. Health and technology. A little writing and creative work. These are the only things I talk about.

'What do you think of the US?' I asked.

'It's great,' Cam said.

'I've been thinking of going.'

'You can always go, it will always be there.'

I didn't tell him I already had a flight booked.

I was secretly hoping he'd tell me to go. I was scared of going on my own decision, I wanted someone else to back me up.

We kept talking.

'What are you after?' Cam asked.

Mike told me to be specific. He was right. Everyone's time is valuable, always be specific.

'Experience in the real world,' I said, 'I've been learning these skills online and I believe I can help your team.'

'What's your week like?', Cam asked, 'can you come in on Thursday?'

'I'll be there.'


Thursday came around.

I was scared again.

Then I remembered someone telling me being scared uses the same hormones as being excited. I flipped the switch in my mind.

I was excited. And I should've been. I'd be given a chance at an internship.

So I made a video about it.

When I first walked in it was so quiet. 'I can't work here,' I said to myself.

'This isn't me.'

I like talking to people. A lot. The quietness wasn't my jam.

Then the day went on, and I realised it was quiet because people were working. Working hard on difficult things. That's what I wanted to be doing.

'Oh, so this is how it works,' I thought.

When I was at Apple, there was never a quiet moment. Lots of busy work rather than deep work.

I went home.

It went better than expected. As always. Everything turns out better than expected.

The next two Thursday's went the same. I was working with Athon and Conor manipulating some data.

'Dan, Athon, how about we go for a walk,' Nick, the CEO, said.

We went to the coffee shop down the road.

'I'm all for them, I'm not a fan of driving,' I said.

We talked about self-driving cars for 30-minutes. Then it died off.

'So, how have you been liking it?' Nick asked.

'It's been incredible, I've been learning so much.'

'You've been doing some great work,' Athon added.

Athon was the machine learning team lead. Hearing that made me feel good.

'How would you like to come on board,' Nick asked, 'into a paid position?'

I wasn't being paid for the internship. Which was fine. The experience was payment enough.

'Let's do it,' I said.

'Okay, sweet, well we'll get an offer together and get it to you by Monday.'

We shook hands and went back to the office.


Around the same time, I was talking with Sal. He was the founder of another company.

Mike introduced me to him as well.

Sal and I were having coffee.

'We'd like someone like you on board to help build a community around what we're working on,' Sal said.

Sal had checked out the work I had been doing online.

I was making YouTube videos about what I was learning as well as plenty of posts elsewhere, LinkedIn, Medium, GitHub.

I'd turned into a human frozen yoghurt machine. All output.

'How about you come in on Friday and meet the team?'

'I'll be there.'

I went in on Friday evening and met the team. They were great.

'Thanks for tonight,' I said.

'No worries,' Sal said, 'I'll be in touch on Monday.'

Monday was the day Nick was going to call me with an offer.


Monday came.

I missed Nick's first call. My phone was on Do Not Disturb. It's set automatically every morning until 12 pm so I can study without being distracted. Probably not the best move when you're expecting a job offer call.

I called back.

'Morning mate, how was the weekend?' Nick asked.

I told him and asked about his.

'We'd like to bring you on as a junior developer at $X per hour,' he said.

'Wow, that's awesome,' I was stoked. Then I remembered someone told me you can always negotiate the first offer.

'I'm definitely interested, but how much wiggle room is there on the rate?'

'What were you thinking?'

'I know I can help your team, with what I've been learning in media as well as artificial intelligence, so I was thinking somewhere around $X.'

The key with counter-offering is to make it about the other person. Instead of what they can do for you, what can you do for them?

My counter offer was 30% higher than the original.

'Okay, I'll discuss it with the others and see what we can do, then get back to you this afternoon.'

'Sounds great! Talk soon,' I said.

We hung up.

My heart was racing. I'd never negotiated pay before. Especially before even having a role.

I knew I would've regretted if I didn't. So I had too. If it scares me, I have to do it.

Nick called back.

'Hey Daniel, we've thought about your offer and would like to meet you halfway at $X.'

The new offer was 15% higher than the original.

'Done. I'm in.'

'Can you come in on Wednesday?'

'Yep, see you then.'


I went in on Wednesday. I was nervous (excited) again.

It was happening. I was starting the role I had been studying for. All much faster than I thought.

I made a video about it.

My first day was April 25. 6-weeks before I was supposed to be jet-setting.

My flight to the US was booked for July 6.

I told Sal I took a role at Max Kelsen.

'All the best Daniel!'

I didn't cancel the flight because I wasn't sure whether I'd stay.

I was in two minds.

One part of me wanted to see where the new role would take me.

The other part of me didn't want to be 43 and saying, 'I really should've gone to the US when I had the chance.'

Now I'm writing this from a cafe in San Francisco.

I didn't cancel the flight. I rescheduled it to September 12. And booked a return flight.

And then yesterday, I visited LinkedIn's San Francisco Headquarters. What an epic place.

LinkedIn_Offices.png

I'll be touring around the US for a couple weeks before heading back to Brisbane to work on a new project we've got at Max Kelsen.

We're going to be using artificial intelligence and genome data to try and predict the outcome of immunotherapy treatment in cancer patients. The project is called Immunotherapy Outcome Prediction or IOP for short.

The crossover of health and technology. Exactly where I want to be.


I had three job offers and a new job within four days of deciding I was going to stop driving Uber. Then a new job within three weeks.

How did this happen?

I have no idea.

Well no, I have a bit of an idea. But I'll never be able to explain it for sure.

When I made the decision to stop driving Uber, I devoted everything towards finding a different role.

I was posting on LinkedIn every day. Reaching out to people multiple times per day.

But it wasn't this alone which helped.

The past year I had been studying hard and sharing my work online. I hadn't been paid a cent for the articles I wrote, the videos I made.

I knew the way I was studying wasn't conventional. I wasn't going to end up with an official Masters Degree. So I needed a way to differentiate myself.

That's when the human frozen yoghurt machine began.

Instead of consuming, I began creating. Every day, I had to create something otherwise I would feel sick.

Someone smarter than me told me, 'It's not what you know, it's who you know.'

Then someone else said 'It's not who you know, it's who knows you.'

So I needed a way for people to know me. And the internet provided it.

The light bulb moment was when I realised I could use platforms I was already browsing every day to create the life I was after.

But the thing is, no one calls you up and tells you this.

YouTube is there but the CEO isn't going to call you and ask you to make videos.

LinkedIn is there but the hiring manager of your favourite company isn't going to say, 'Hey you should be posting more often, it could end up in you getting a job here.'

Everyone can blog but no one is going to tell you to write about what interests you.

You have to choose yourself.

I chose myself.

People had a way of finding out who I was.

Then instead of being a resume in a pile, I was a person.

People relate better to people. It's hard to make a connection to a piece of paper with a name and a list of credentials.

A year ago, the web startup we built failed. So I decided to start learning machine learning.

I had no job and university was too expensive so I wrote my own curriculum.

I worked through it Monday-Friday and drove Uber on weekends to pay for courses. And repeated this week in, week out.

Until one day I grabbed my balls and decided it was time to stop driving Uber and chase after what I really wanted.

And then the other day, I received these.

Max_Kelsen_Business_Cards.png

I began reaching out to people on LinkedIn because I didn't know what to do. So I asked for advice.

I asked Cam and Mike what they thought about the US because I was unsure of how it would turn out.

Now I'm in the US and more excited for the future than ever. But I still don't know how it will work out.

Well, what's next?

I don't know. I don't think too far ahead.

Instead, I try make every day a good day.

A good day involves learning something, creating something, moving my body and talking to the people I love.

String enough good days together and you've got a good week. A few good weeks makes a good month. A dozen good months equals a good year. Enough good years and you'll end up living a good life.

Doing the things you set out to do always feels good. And even if you fail, you'll have a good story.

Choose yourself for a year. The whole universe will get behind you. And you'll be surprised where you end up.And if it's in San Francisco, I recommend Blue Bottle Coffee.

Getting that parking fine is the best thing that's happened to me all year.

Source: http://qr.ae/TUN3oh

How do you learn machine learning?

Agh.

There’s so much going on.

Dozens of papers every week.

New releases from Facebook and Google every week which send the media into a frenzy.

Articles on the web showing the latest and greatest courses you should be doing.

Where to start?

A year ago, I was asking myself the same question. I quit my job to start a web startup with my friends. It failed. But along the way, I was hearing about this machine learning thing.

‘The computer can learn things for you?’ I said to myself. I knew I had to get involved but didn’t know how.

So I made my own curriculum to at least have some structure.

And then yesterday, I received these.

IMG_2063.png

I know role titles don’t account for much but this was a pretty big milestone for me. I set myself a goal and worked towards it.

I’m still working towards it.

You can too.


A) START WITH WHY

All those courses you start but never finished. Why?

All those things you said you’d do but didn’t. Why?

It’s likely you didn’t have a strong enough reason to begin with.

Before you start learning machine learning before you start learning anything. Ask yourself, ‘why?’

‘Why do I want to learn these skills?’

Find a pen and some paper. Then take a walk. Alone.

Sit down at a nice quiet place.

Take the pen to the top of the page.

‘I want to learn machine learning because...’

Then write down everything that comes to mind.

‘I want a better salary.’

‘I want to build apps to help others.’

‘The technology fascinates me.’

‘I want to be a part of building the future.’

Anything goes. There’s no right or wrong answer. Write it all down.

Got a full page? Good.

Now read back over them. Are they enough to keep you going for the next 10-years?

Why 10-years? Because why sign up to anything if you’re not willing to commit to it?

If they’re not enough to keep you going for 10-years, find something else to learn.

If they are, tape the piece of paper to your wall. You’ve now got your why.

Every time studying gets hard or you think you’re not good enough, refer back to your why.


B) LONG TERM

So. Many. Papers.

I study this stuff every day. And I still can’t keep up.

It takes me 6-hours to properly read a paper. That’s less than 10 a week.

Every couple of weeks there’s a new benchmark. A new way of doing things. A new model to try. A new network architecture which does 1% better than the last.

It’s easy to get lost paying attention to all of this. It can be daunting.

‘Why can’t I ever keep up with what’s going on?’

You shouldn’t not ever pay attention to what’s new. It’s important to know how the field is progressing. But to think you’ll be able to keep up with every single detail, especially when you’re starting out is stupid.

Trying to keep up with everything new will hold you back.

Think long term. The main drivers of machine learning have been around for decades. These aren’t going away anytime soon.

  • Programming (Python, R, Java) - Pick a language and stick with it. Personally, I’m a Python fan but it’s because I’ve never tried anything else.
  • Mathematics (linear algebra, calculus, matrix manipulation, optimisation, statistics) - The highest math education I’ve had is high school since then, I’ve been using Khan Academy to improve my skills.
  • Computing (local hardware, cloud computing, parallelism) - The cloud is abstracting away much of what you need to know about this but if you’re starting out, at least figure out how to activate a computer on the cloud.
  • Communication (what do you know and how can that benefit others?) - Communication is valuable everywhere. Your skills are no good if you can’t explain them to the person next to you.

Work on building a foundation of knowledge and skills around these topics. Then when something new comes out, you’ve got a solid platform to launch off.

Don’t expect to happen overnight. Getting good at these skills takes time and effort. Think months and years not days and weeks. There’s a reason machine learning skills are in demand.

When it gets hard, and it will refer back to A.


C) TRADITIONAL VS. NON-TRADITIONAL

Ron and I were looking up a Bachelor of Mathematics yesterday. 3-years with a major in Applied Mathematics and a minor in Bioinformatics — I love health.

The majority of my learning has been online but after spending a few months in the industry I’ve caught the learning bug.

I’m hungry to learn more. And math is everywhere. It’s the language of nature.

I graduated in 2015 with a Bachelor of Science, Dual Major in Food Science and Nutrition.

My why is to combine my love for health and technology. I don’t know how yet but it’ll come.

A few months after graduating, I went to an open day at my university.

‘How much is a Masters of Computer Science?’ I asked.

‘$42,000 per year,’ she said, ‘with a half payment up front.’

‘What can I study?’

She handed me the brochure.

I read through it and didn’t understand half the course names.

‘Thanks,’ I said.

I went home and consulted my mentor. Google is always there for me.

I typed ‘learn programming online’ or something of the like in the search bar.

Back came some results. I clicked on the first one.

It was Udacity. I spent over an hour looking through their offerings. The pretty colours lured me in.

Seeing all the things I could learn online was exciting. And all for far less than what I was quoted at the open day. I closed the browser.

Then it was a while later and we were building a website and I needed to learn something. Up came Udacity again. There were more offerings. This time with an entertaining character describing deep learning. Siraj Raval knows how to educate and entertain at the same time — the best way to teach.

I signed up for Udacity’s Deep Learning Nanodegree. It started in 3-weeks but I’d never done any Python. So I signed up for Treehouse’s Python Track.

It was hard but my why was strong enough. I was driving Uber on the weekends to pay for my courses. I had two whys. Combine technology and health and stop driving Uber.

After finishing the Deep Learning Nanodegree, I was a little lost. So I went back to my mentor and asked for some guidance. I really liked Udacity’s style of teaching so I found similar courses online and combined them to make my own AI Masters Degree.

I’m still going with it. It’s been a great challenge.

What’s the advantage of studying online?

  • It’s fulfilling to set out your own path and work through it.
  • It’s cheaper than university.
  • You can study when you want — no compulsory lectures.
  • You can study anywhere you want — no longer do you have to physically be at the location to learn from the best in the world.
  • You can learn exactly what you want — course not living up to your expectations? Choose another.

‘Alright, Daniel, all that sounds great but what are the downsides?’

  • Many hours spent alone in your room — it can get lonely sometimes.
  • No one is going to check up on you — you’re it.
  • No shiny certificate at the end — sure, digital certificates are great but real degrees still have more prestige.

To work around the downside of studying alone, I started sharing my journey online. So I’d find more people like me. And I found them. It’s great talking to people on a similar path. We help each other.

I’m a big promoter of online study because it’s what I’ve done. And the educational resources available there are growing every day.

But are you better off going down a more traditional route?

It took me 5-years to do a 3-year degree. I failed my first 2-years. I figured I’d spent enough time at university this decade.

Choose which is best for you.

Does your why require you to have a degree?

Would it help you reach your goals?

Or can you achieve what you want to achieve without going to university?

If you want to get into machine learning research and pursue a PhD program, university is probably the best for you.

If you learn best around a cohort of others learning the same things as you, university is probably the right option.

If you’re self-disciplined, can learn things on your own, and can’t afford university, you’ve got the internet.

What I really did during my first 5-years at university was to learn how to learn. Now I feel like I can learn anything.

All it takes is sustained effort over time.

There’s no best way to learn something other than the way that keeps you wanting more.

For reference, here are some of the courses I’ve done.

You can see more in my full curriculum.


D) SHOW DON’T TELL

I knew my method of learning wouldn’t be recognised as much as more traditional options.

I had to do something to showcase where I could bring value.

I’ll never be the best engineer. But where I can bring value is by being a translator.

The communicative bridge between engineers and customers.

To get skin in the game, I started making videos about what I was learning. And writing articles on the same.

Be honest with yourself. And not only about your weaknesses. It’s easy to be your own biggest critique. Be honest about your strengths too. And show them, don’t just tell people about them.

If you’re a great engineer, show your code, explain it to others.

If you’re brilliant at math, write an article breaking down a proof.

My cards say machine learning engineer but if I think about where I can bring the most value in the future, it’ll be as the front man. I’ll help the people smarter than me communicate how their work can benefit the world.

Not good at anything yet? Show that too.

It’ll help you figure out what you need to improve on.

Show don’t tell.


E) BE THE DUMBEST

When people introduced themselves in the Deep Learning Slack channel it scared me.

‘Hi everyone, I’m Paul, I’m a machine learning engineer at Google.’

‘Hello, my name is Sandra, I’m a software developer at Big Software Company & Co.’

Here I was 3-weeks into learning Python.

‘How am I going to ask a question?’ I thought to myself, ‘everyone is going think I’m dumb!’

Then I realised, being the dumbest in the room meant I could learn the most. If you’re the smartest person in the room, you’re in the wrong room.

I was in a room the other day with one of the lead Genomics researchers in the world. We’re doing a joint project with her team. And she told us, ‘the only stupid question is the one that isn’t asked.’

Don’t be like I was and afraid to ask ‘stupid’ questions.


F) DRIVER VS. MECHANIC

When you drive to the store, how often do you think about how much torque the engine is producing?

Or the amount of air flowing through it?

Or how the fuel pump delivers fuel to the combustion chamber?

A car can be useful to you without this knowledge.

However, if you want to be a mechanic, knowing about these things is a good idea.

The same goes for machine learning. You can learn how to use programming frameworks and libraries such as TensorFlow to build things which are useful without actually having to know what’s going on under the hood.

But if you want to be able to make your networks train faster, work better on different devices, be more computationally efficient, you’re going to have to get some grease on your jacket.

In the beginning, focus on being a driver. Get yourself around. Like driving from one store to the next, solve one machine learning problem after another.

When you’re comfortable driving the car you have, it’s time to upgrade, pop open the hood and dig a little deeper. Practice on a new problem you’ve never seen before. One where you don’t what the answer should look like.

The best thing? If you mess something up you can always reverse the code. There isn’t version control for car engines yet.


G) NON-STOP

The learning never stops.

Every day is day one. When people ask what I do, I still reply with, ‘I’m a student.’

What’s the alternative?

Keep the same knowledge for the next few decades?

No thanks.

The more you learn, the more you realise you don’t know.

My maths skills aren’t as good as they could be.

My statistics knowledge isn’t top notch.

My programming is mediocre at best.

Communication is pretty good. But there’s always room for improvement.

All the real benefits in life come from compound interest. Especially knowledge. But it’s hard to realise it in the moment.

You could study all weekend and no one would notice or care. But you would. You refer back to your why.

Lay one knowledge brick a day and your future self with thank you for the beautiful library you’ve built.


H) TL;DR

1. Start with why. Write your reasons down and make them concrete. When it gets hard. And it will. Refer back to that piece of paper you stuck to your wall.

2. In the beginning, focus on long-term knowledge gains rather than the flavour of the month.

3. Do you need that university degree? Or can you get the skills you’re after without it?

4. Showcase your work. Even if it’s bad. Explain why you did what you did. And how others can avoid your mistakes. Even the best products won’t sell if they don’t have any shelf space.

5. Ask questions. Find people who have been through what you’re going through and ask them for advice. You don’t have to listen to it all, remix it with your own and make it better.

6. You don’t have to be a mechanic to drive a car. Take your new knowledge for a test drive as soon as possible. Don’t let it sit there going to waste in the archives of your brain.

7. The learning stops when your heart does. Keep learning.


I) BONUS

When I first started as a machine learning engineer, what we worked on day to day was very different to what I had in my head.

It’s not always running super sophisticated models on giant computers in the cloud. That’s only the tip of the iceberg.

I made a video showcasing some of the things we get up to on Monday’s at Max Kelsen. Monday’s are research days, where we try things that might not work.

If you’ve got any questions, reach out anytime. You’ll find me everywhere online.

Back to looking more into that Bachelor of Mathematics. I’ll let you know if I end up doing it.

Source: http://qr.ae/TUNima

Talking AI to a group of Chinese academics with Athon

I was scared when I first read the email. Presenting to a group of Chinese academics. Was I ready for it?

"This is a stretch even for me," I thought.

But what was the worst that could happen? The talk doesn't go so well?

No matter, I'd be back where I am right now soon enough. Back in bed. Where I'm writing this post from.

Last Thursday, I received an email asking me to come and present what we have been working on at Max Kelsen. Particularly our work on AI in healthcare and other areas.

I asked Athon if he wanted to come along. He said yes. He's a great team leader. Plus, I thought it would be good to have some real technical knowledge along for the show.

We confirmed the talk for Monday. We had nothing ready but the knowledge was in our heads, all we had to do was present it.

I spent Saturday and Sunday preparing. Monday morning I finished off the slides. My part was AI in healthcare. I'm fascinated by it. Ever since my mum told me about bionic man and woman as a kid.

The talk went better than we thought. It's always the case. Most of our worst fears are inside our heads and all of them are of the unknown. I've done plenty of public speaking before but never to a crowd like this.

We recorded the talk. I went into details about how there's an information gap in healthcare. The current system works on episodic and reactive treatments rather than continuous and proactive. For good AI work to be done, this information gap needs to be closed. How? Through combining genomics, clinical information and sensor information to gather insights on a persons health and then infer future outcomes.

Athon talked about some of the amazing work he and the rest of the team have done at Max Kelsen. They built an image recogniser which uses satellite images to figure out whether a site is a construction site or not. This helps the city council with planning. And a tool which helps lawyers decide if a case is worth taking on.

I'm lucky to have landed a role at Max Kelsen. We're working on some great stuff.

All areas of AI fascinate me but healthcare is closest to my heart. Plenty more content and work around these two to come.

I'll post the full video on my YouTube when it's edited. Margeuax, the organiser of the talk is sending me the audio recorded from the room. It may be a little better than my iPhone.

 Was it really an AI talk if you didn't show a picture of a GPU?

Was it really an AI talk if you didn't show a picture of a GPU?

Using Deep Reinforcement Learning to Play Sonic the Hedgehog

In between projects over the past couple of weeks, the Max Kelsen team and I attempted to replicate the epic World Models architecture to play Sonic the Hedgehog for the OpenAI Retro Contest.

Although our algorithm wasn't fully ready by the submission date, we still learned a great deal in the process.

The full writeup of our approach is available on Medium.