Short list of ML-Ops resources
There is increasing the need for software engineering practices to be applied to ML. This is the field of MLops.
This is a field I want to learn about. A redditor noticed that that the area seems a bit fragmented. And does not know what combinations would be suitable for use case.
I know very little about the topic. But I will provide some links I’ve seen, and I want to use. To learn about the subject.
Full-stack deep learning: A course that focuses on the production side of ML.
DVCorg ML-Ops tutorials: A YouTube playlist showing how to use GitHub actions for ml ops
Github actions for ML-ops: A blog post from GitHub showing how GitHub actions can be used for ML-ops and data science
MLOps Tooling Landscape v2 (+84 new tools) - Dec '20: A decent rundown of the ML-Ops field. (Also follow the author she writes regularly about ML-ops. Chip Huyen)
How to Deploy a Machine Learning Model to Google Cloud for 20% Software Engineers (CS329s tutorial): Example of deploying a model
GitHub for MLOps: Collection of blog posts using GitHub for ML-ops
How to improve software engineering skills as a researcher: A guide showing how to use software engineering tools for your deep learning model.
More Suggestions from Redditors:
Machine Learning Engineering for Production (MLOps) Specialization: Course by Andrew Ng about MLops
Gokul Mohandas Tutorial: MLOps course teaching the beginners about ML-ops
AI Engineering Youtube playlist: Youtube playlist about MLOps
DataTalks.Club DataOps: Podcast episode about DataOps
DataTalks.Club MLOps: Podcast episode about MLOPs
--
I haven’t used them personally, but people have given these resources good reviews.
Honestly, I want to release couple of my projects outside a Google Colab notebook. Also learning about ML ops seems helpful for industry experience. (I don’t know, please correct me if I’m wrong)
If you like more compilations of resources like this, Sign up for my mailing list.
Tips For Learning ML If You Don't Like learning Via Courses
I read a Reddit post about how the OP was struggling to learn ML. Because he found the courses abstract and theoretical. And did not see how it would relate to his ML goals. In the people gave their opinions and useful suggestions.
Some of those suggestions I will be showing below:
Create A Side Project
Start working on a project that would involve ML. Then you can learn about the topic as you’re developing the project. You can write about what learned in a blog post, so you know what to work on next time.
Implement A Paper
Implementing a paper helps learn new concepts and forces you to translate that knowledge into a tangible item.
Take Courses That Focus On Coding Models Straight Away
I recommend FastAI which is a very hands-on course. Which focuses on working on ML examples straight away. This course helps you learn the basics of Deep Learning while of some tangible examples to show.
Tutorials Provided By PyTorch and Tensorflow
You can try the tutorials provided on their websites. You will work through practical examples on how to use the library. And you can read about the concepts that some tutorials talk about along the way.
Create Your Favourite Models From Scratch
This idea is from Andrew Trask. You create neural networks only using NumPy. This will force you to turn any theoretical knowledge you have about ML into real-life examples. It won’t be enough to name a concept and move on. But you will need to make tangible examples of the concepts. This can be done for your favourite libraries as well.
Additional note:
You still need theoretical knowledge if you want to do well with ML. As want to know how your model works behind the scenes. And it helps you grasp any new concept that comes your way. If you want to learn about maths. Check out these resources (MML book and YC Learning Math for Machine Learning). As maths is something that many people struggle with when learning ML.
After this, you should be more confident about learning ML. And have hands-on experience making models and a greater understanding of courses you were watching.
-
If you found this post useful, then check out my mailing list where I write more stuff like this.
If tech CEOs are mayors then why do we expected so much from them?
I was reading an article from the newsletter called the Diff. Which the author gave the analogy that a tech CEO is an overworked mayor. Running a platform is like running a city. As your not just setting a price for your product. But deciding how your platform should be governed.
The newsletter argues:
Moreover, the problems Facebook has to solve are not the cosmic, inspiring ones. They’re pretty trivial, technocratic issues, mostly dealing with competing interest groups rather than competing ideologies. Zuckerberg isn’t an emperor, or even a prime minister; he’s the world’s most competent and most overworked mayor. The question of how much data an advertiser should be able to collect and use, and how they should be able to use it, isn’t a question with the same scope as a treaty or a labor law; it’s a lot more like deciding where a sewage treatment plant goes or choosing which bus route to cut.
This is understandable, platform businesses are more about choosing regulations and how to implement them. But Facebook or Twitter are not signing treaties or bills on these rules. But simply juggling between interest groups over an decision. Reminds me of the YouTube situation. In which YouTube must manage the interests of creators and the advertisers.
With other important interest groups like corporate entertainment and news companies. These interest groups bring YouTube money with adverting. So YouTube tends to give this group preferential treatment. Which YouTube creators deem unfair. Deciding when to notify when somebody uploads looks more like how the bin service should run.
Deciding to send a notification for a new upload. Looks closer to people arguing how bin service should be run. With different neighbourhoods lobbying for better service. Than the drafting of the Maastricht treaty.
Many of the social media problems are strictly practical matters. How do you regulate hate speech? What practices should you adopt to judge content? In twitter’s case do you have enough tools to even deal with the problem?
I'm starting to learn that a platform/marketplace business is a double-edged sword. Yes, you run the casino. So, you get the biggest profits out of the rest of the tech companies. But you have a lot of responsibility. With people expecting you to do magic with limited resources. Or in Facebook’s case make decisions with no good trade-offs.
Do you think going to congress is fun, while both sides are yelling at you are not doing enough? While the congresspeople have no clue what they are talking about.
Jack Dorsey made a joke about this in the recent hearings. On Twitter he made a poll with a simple question mark. With the answer yes and no. A clear jab at the type of questions the congresspeople was asking.
The termination of the ex-president was a hard decision to make. Where the tech leaders had to take into account their progressive employees, the legal liability of a person spouting falsehoods or even violence. Also considering the advertisers paying the platforms. And the fans of the former president.
When you have so many interest groups, some people are bound to lose out. In many companies, your stakeholders can be the shareholders, employees and customers. Some public companies only need to worry about the shareholders.
The article touches on this subject:
GM needs to balance the interests of drivers, dealers, employees, suppliers, and shareholders, but basically all of those groups either want to get more money or spend less money, so GM has the comparatively simple option of building a really good product. But if Facebook builds a really good product for spreading news, they’ve built an exceptional product for sharing fake news; if Google has a good way to surface information, it’s also a good way to surface misinformation.
Jack Dorsey and Mark Zuckerberg only get the final say because these decisions are very hard. Facebook is forced to follow local government instructions to remove material from its website. This can be normal stuff like abuse or explicit material. Or political sensitive stuff like removing opposition leaders of the local government, political groups etc.
One example I read on ProPublica was Facebook was removing some Kurdish material with advice from the Turkish government. What was interesting was the internal emails of the thinking. Which tried their best not remove the content. But once it became clear that a ban was imminent. Then opted for a geo-block of the page. Twitter did something similar with the Indian farmers. Geo-blocking best of the bad options when it comes to censorship. As the affected country can’t see the content but everybody else can. Also people with VPNs can still access it.
Removing content that does not cause harm is not something they want to do. As they are American companies with culture of free speech. But governments are an important interest groups that can’t be ignored. If not, they will be shut out of dozens of countries. And miss out on millions of users.
The decision to geoblock content rather than remove the material outright is like working out how to enforce the law in a city. Some people want a no-mercy style approach others to want a selected approach where only serious crime is prosecuted. Your job as the mayor is to take all of these interests and approaches into account. And make an optimal decision using given knowledge.
[use another example]
America tends to take a pretty relaxed approach to regulation. In China tech companies are always under the scrutiny of the government. In an article from the Protocol[insert link], a worker from ByteDance explained that the company had to follow directives from the government. Which called to censor certain words and topics.
It was up to ByteDance to implement the directives:
During livestreaming shows, every audio clip would be automatically transcribed into text, allowing algorithms to compare the notes with a long and constantly-updated list of sensitive words, dates and names, as well as Natural Language Processing models. Algorithms would then analyze whether the content was risky enough to require individual monitoring.
To follow the rules there is a lot of work done behind the scenes to make it possible. Just like a mayor will need to make new housing developments follow building regulations. Bytedance needs to find a way to follow the rules or the government will take action.
Being emperor of Facebook or Twitter does not seem fun does it?
The author notes:
This is the problem of platforms: they build a place, rather than a business, so they can’t enapsulate complexity by making everything transactional. The upside to this model is that it leads to long-term, high-margin growth. Building a platform means homesteading a new economic frontier, and then running it as an idealized government that taxes at the Laffer maximum and most lets participants alone.
Being a tech CEO is hard. Because your not just selling weights. But dealing with numerous people with competing interests. People will always have problems. So tech companies always in a cycle of dealing with demands.
From the newsletter:
Unfortunately, new property rights require an immense and tedious investment in codification. When you’ve solved product, sales, marketing, and operations, the only thing left is politics, and by its nature, politics doesn’t get solved.
How to run python scripts on Google Colab
Did you know that you run python scripts in Google Colab?
No? Neither did I.
Just learned from reading a reddit comment.
I’m going to show you how to do so in this blog post. So you use Google’s compute power to its full use.
If I learned about this earlier then many of my projects would make be easier.
If you saved your script in your google drive folder. Then click the mont google drive button. And provide permission.
I’m just going to have a simple hello world script:
print('Hello World, This is a Google Colab script')Uploading locally, you can click the upload to session storage. Should work if you file is not too large.
Then you run this statement:
!python '/content/colab_script.py'With the result:
Hello World, This is a Google Colab scriptYou can upload to drive using:
from google.colab import files uploaded = files.upload() for fn in uploaded.keys(): print('User uploaded file "{name}" with length {length} bytes'.format( name=fn, length=len(uploaded[fn])))
First the file will be saved in session storage then you can move it into your drive folder.
NOTE: I don’t know if this is a bug. But uploading your py file via the normal file upload on google drive (not colab). Turns the py file into a gdoc file. Which google colab can’t run. So you one will need to upload your file though google colab. To access your files.
Hopefully you found this useful. Knowing you can run some of your scripts on Google Colab.
Where does creating a network and country begin?
I was reading an interview by the musical.ly co-founder and now VP of TikTok Alex Zhu. Describing how creating a social network is like making a new country.
This is how he describes it:
Building an influencer community is very similar to building a new country (economy) from the ground up. In the early stage, building a community from scratch is a lot like discovering new land. Imagine you just discover new land. Let's call it America. Now you want to build an economy. You want to grow the population and you want people to migrate to your country.
Creating something out of nothing is no small feat. You need to answer questions like: How do you create jobs with nothing there? How do you get more people to migrate to your country?
Right now I'm rereading the biography of Lee Kuan Yew the founder of modern Singapore. And something the early chapters remind me of this discussion that Alex Zhu is having. LKY worried a lot about the question of creating an economy. How to create new jobs when you have a new country. While some of Singapore's worries do not translate well into creating a new social media app. Like the question of defence. Being worried about invasions by neighbours or coups. Are not too relevant when getting people to your social media platform. The closest analogy to that is Facebook creating a clone to attack your network. But they have a less than optimal hit rate.
It makes me wonder – Where is the line of creating a new social network vs a country begin?
I found the Alex Zhu example from Li Jin. A great thinker of the creator economy in her article she laid down details on how to improve the creator economy to have some type of middle class. Almost like an American dream for the internet age.
Making people feel they have a shot at success, is the two things being a country and social networks have in common.
But it’s a chicken and egg scenario. How to create a system of value with nothing there?
Alex Zhu gives the analogy between Europe and the new land America of the 1800s.
Let use an analogy: Musical.ly will be America in this analogy, and YouTube/Instagram will be Europe. How do you convince creators from other regions (social platforms) to move to America (Musical.ly)?
The problem with Europe (YouTube & Instagram) is that the social class is already well-established. The average citizen of Europe has almost zero opportunity to move upward in the social class. We saw an opportunity to leverage this. We will build for the average citizen in Europe.
I wrote about this before. YouTube and Instagram lack the dynamism that it once had.
Because of that is harder for newcomers to join the ranks. The algorithm of those platforms favours the incumbents. As they know they can draw in a large crowd. Compared to an untested newcomer. This does not mean newcomers never succeed in those systems. But it’s much harder.
TikTok is famous for allowing newcomers to join the service and allowing them to succeed. There are many stories of TikTokers that only have less than 10 videos going viral. This is much less likely with Instagram and YouTube.
Granted, the design of the service makes life easier. Dealing with 10-second videos makes it easier for the algorithm. To go through tons of examples compared to a 10-minute YouTube video. If you are interested in this then check out Eugene Wei Remains of the Day blog post.
Alex also had some more details about the matter:
In this new land, you have to build a centralized economy in the early days. This means that wealth distribution is accruing to a small percentage of people in your land. You make sure they successfully build an audience and wealth. This makes them role models for the country (and platform). You effectively create the American dream. People in Europe (Instagram) will start to realize that this "normal" person went to America (Musical.ly) and became super-rich. Maybe I can do the same? This will lead to a lot of people migrating to your country (platform).
Here Alex is explaining that you want to have a heavy hand in the creation of the platform. In country terms, the government will be the guiding factor in the economy. Examples are having state-owned companies and subsidies. The Asian tigers [insert link of book] did this in the early years of development. With a large state calling the shots of the national economy.
In this passage, Alex explained how you want role models for your platform. So other people can see the rules and the culture of the platform. In country terms, this will promote some companies and industries over others. And Alex explains if you are doing right then people from other platforms or other countries will be willing to emigrate to the new platform. Due to the chance of success. Starting a flywheel of great talent entering the platform or country.
Therefore, once countries become successful it is hard for them to stop. Unless a major crisis happens and that is a maybe. Then the country will start to decline. In social media terms, this is the power of network effects. Were the flywheel starts spinning. And it’s hard to stop. Only after terrible mismanagement. Or an even better platform (the platform needs to be 10x better though). Then people will move away from the platform.
From reading the Lee Kuan Yew biography and this description of musical.ly and now TikTok. Is that anything great will be very hard in the early days or years. Lee Kwan Yew had to worry about creating a country that had jobs to avoid choric unemployment. Also, build an army so local neighbours don’t invade. Tiktok had to get enough users to start the flywheel. And make sure it does not get crushed by a big tech rival like Facebook.
Tech Is Not Just Bad nor Good: Biological Determinism In Tech
Biological Determinism
Last week I read a report by the Data & Society. A research group that investigates the impact of technology. The report talks about the common myths of technology. One myth I will be exploring is social media is addictive and powerless to resist.
They examined that most myths in the report come down to biological determinism:
which suggests that our “Paleolithic” brains cannot resist “God-like” technology, placing too much power in the hands of tech companies to both create and destroy our capacity for attention.
This concept gives a good idea of the recent anti-tech movement.
The general idea here is we don’t give much credit to ourselves when dealing with technology. They are people that have healthy relationships with their devices. So it must be possible.
When explaining the myth in the report. They mention that behaviours that we moralise and put a biological spin. Make us forget about human agency and cultural surroundings.
Then went on to explain that new technologies tend to have serious moral panic. Normally with concern about women and children.
The pessimist archive talked about this a lot. With the funny examples of books and bicycles. The account took a parting shot at the most famous materials of technology moral panic today.
The Social Dilemma.
A documentary about ex-big tech workers talking about the ills of technology. In the documentary the host, Tristan Harris said there was no moral panic against bicycles. He was wrong. The pessimists' archive overlayed headlines while talking about the topic. Funny video indeed.
https://twitter.com/PessimistsArc/status/1306011195212783618?s=20
The report mentions:
This myth reinforces the narrative that technology design leads to control of millions of users, locating enormous power with a small group of tech companies.
Nir Eyal touches on this as well. That technology addiction has very little evidence. So this is an area that needs to have more research. But for now, we should not be too sure about ourselves. When it comes to this topic.
We Are Not All The Same
The report mentions that different people have different responses to technology. Which is obvious when you think about it. But gets lost in our discourse. Different mental health and wellbeing can be different for young adults using social media.
Different groups use platforms differently. Marginalized communities may benefit from increased access, visibility, and community through these platforms.
The conclusion of the section explains that you should not assume everybody using tech is going to get addicted. Nor have mental health issues.
They called for more expansion of research. Meaning people from different background be included in the research and product designs.
In many areas of the world being part of the LGBT community can lead to persecution. So finding a community online. Can be the only way to find like-minded people. And talk about the issues they deal with.
In some areas of the world. Access to certain information is very difficult. If you’re an Iranian that wants to watch Hollywood movies. One will need to pirate the films. Due to an embargo of the country.
Yes, we don’t need to be on our phones 24/7 but they are some very good uses for using our devices.
Even society may change, which forces you to use technology differently. Coronavirus is the best example of that. In which people learnt about remote work. And got accustomed to using zoom and other workplace software.
Your laptop was primarily used to watch Netflix and YouTube. Now you are doing work instead. You may be spending the same hours on the device. But doing completely different tasks.
This is why we need nuance when talking about the downsides of tech. They are important to talk about. So we can improve devices and reduce the harm created by them. But hyperbole does not help.
Even with that example. They can still be issues. Many people have the problem of their workplace being always on. The employee always needs to be on call. Leaving not much time for anything outside of work. So, you can never turn off from work.
Cal Newport talks about this a lot in this blog. About the issue of people always being plugged into the workplace. Without getting much work done. Which he calls the hyperactive hive mind.
While this is a serious issue. Do we say people are addicted to zoom or email?[1]
Talking about how technology can be improved in a certain context will be helpful. Not just labelling a device bad and waiting for the tech companies to do something. Or poorly written legislation to fix it.
Technology Is Not Just Good or Bad (The Need For Nuance)
I like what the report says:
The concept of addiction does not encompass the full range of pleasures, risks, and uses that people create with technology
A summary of what I said. Using the term addiction implies that you are addicted or not, technology is only good or bad. Which is not the case. In some contexts, a certain technology is fantastic but in other cases not so.
Changing what words we use that describes our relationship with technology. May help us get rid of some of the guilt associated with that line of thinking. So when using your phone you don’t feel bad. When you are getting value from the activity.
This does not say you should do nothing with your relationship with tech. Many people recommend turning off notifications on your phone. And don’t recommend using your device late at night. And other activities.
You should ask, what value this device or technology provides for me?
When thinking deeply about this answer. You should not feel bad when using your phone for Netflix. If that’s what you plan to do. And you don’t need to think of yourself as “addicted” to Netflix. Just a consumer who wants to watch good TV shows.
-
If you liked this article. Sign up for my mailing list. Where I write more stuff like this. And a place to share your thoughts on the blog posts.
[1] Some people say your addicted to email, That I may add
The answer to for-profit social media: Digital Public Spaces
I read a WIRED opinion piece. Which talked about the need for public digital spaces. Which is social media not driven by profit.
He called them online parks. Digital spaces where you can have a good time like public parks. But you don’t have billboards anywhere telling you to buy some useless trinkets.
This can resemble real parks where you can meet people of all backgrounds. And move us closer to the goal of an open web.
Right now, you meet people around the world on the internet. But with echo chambers, you start to deal with people that think like you and may even look like you.
Funding the New Public Digital Space
The author explains that social media can’t be done for profit. Because of the incentives that being a for-profit provides.
The funding for the non-profit social media service will be an issue. Which the author did mention in his article. That it may need a wealthy backer in the beginning. Almost like a seed round.
Maybe the service can run on donations at the very start. I don’t know.
Also, there needs to be a good selling point for people to use this service. Just saying the service does not make money. May not be a good selling point. But I could be wrong. Signal is very popular right now. Because they are a non-profit organisation. That promise not to collect your data in exchange for money.
Or maybe one can still have a for-profit service. But will need a new monetisation model. Telegram is a good example of this. The app is free to use. But the company makes money by selling stickers on its platform.
Something similar can be done. The video game equivalent of selling skins. (ex, Fortnite) Some of the Asian messaging apps already do this like KaKaoTalk and LINE.
Maybe the service will still want to sell ads. But without the algorithm promoting vitriol and anger. But that’s a massive known. Due to engaging posts mostly defaulting to negative emotions.
I know people give the idea of having a paid service for Facebook. But the fact is the data helps improve the product. So having a paid product in exchange for not being tracked may not be good as the free version. Uncomfortable that it may be. Also, you want the service to be free so many users sign up as possible. Which will increase the value of the network due to network effects.
What the Digital Public Space Could Look Like
While social media is flawed. It has allowed staying in touch with our friends and family. Anywhere in the world.
Imagine living through the pandemic without tech. It will be almost impossible to talk to your friends and family. If they are a large distance away from you.
Social media allowed people to participate in communities while staying at home. (Some arguably bad).
The closest example of what the service can be is Twitter. Which does not have a big market cap as Facebook. But has massive cultural relevance. Dictates the news cycle. A massive networking tool in many industries. Ground zero for the best memes.
The author said the service should be used for discourse. Just with a lower temperature. The main goal may not be privacy, but it certainly helps.
Having a service that allows you to have many of the functions of social media. Without the vitriol will be a good selling point. But that will require strong moderation. Which will upset a few people.
But I guess you have the right to kick out a person at a park for doing damage or harming residents.
A non-profit or non-VC backed service. Does not need to worry about growth to the N-th degree. Almost breaking their product to get more users. This should avoid many of the problems social media companies have faced. As they are more worried about growing. Than fixing many issues that plagued the service.
Issues Facing This Idea
Like I said earlier. I don’t want to make this into a Facebook and Twitter bashing battle. While they are flawed services. I think the world is better with them.
We just need to focus on iterating these services and social media in general in a better direction.
To better align the original vision of a free and open web.
Nothing can be ever perfect. Telegram and Signal are great messengers. Both have issues of dangerous groups starting to use their platform. A couple of years ago Islamic terrorists were using telegram to share propaganda. Signal is seeing a rise of far-right groups using the service.
So this new service won't get rid of all negativity of the web. And it won't be without its issues. But having people using a service without feeling bad about themselves is a noble goal.
People should be allowed to speak about political topics without tearing apart society. And leading to further polarisation.
This online park idea has a big hill to climb. One will need a good selling point, so people join the platform. One will need a way to sustain itself. One will need a way to deal with many of the ethical issues hitting the platforms right now like misinformation and moderation.
The design of the service must be built in a way. To avoid polarisation and vitriol. That will require a lot of thinking. Many times, issues of trolling and negativity are simply of community size. The larger a community gets the more likely they are to be bad apples. And harder to enforce community rules.
If that’s the case, how can a good experience be scaled up to millions of users?
The service will likely to put safety first and foremost. Not an afterthought, unlike a few companies. Maybe integrated into the product team. So, product decisions can be made with safety in mind.
These are my thoughts on this topic. The article is interesting you should check it out. Provides a different model for social media. I don’t think non-profit social media will fix all the problems, but it can help.
Commercial social media is not all bad. It has allowed millions of people to communicate with each other.
…
If you liked this article. Sign up to my mailing list. Where write more stuff like this.
Using assert statements to prevent errors in ML code
In my previous post, I showed what I learnt about unit testing. Testing tends not to be thought about when coding ML models. (The exception being production). So, I thought it will be an interesting topic to learn about.
I found one unit test to try out because it solves an issue. I face a lot when coding up my model.
The unit test checks if I’m passing the right shape of data into the model. Because I make this simple mistake from time to time. This mistake can add hours to your project. If you don’t know the source of the problem.
After I shared the blog post on Reddit. A Redditor commented. “Why not just use assert?”
That was something that did not cross my mind. So, I rejigged my memory, by checking out what assert did.
Then started working out how to use it for testing ML code.
One of the most popular blog posts on the internet about ML testing. Uses assertion statements to test the code.
When writing an assertion statement making a function is needed most of the time. This is how unit tests can be made.
Assertion Statement for the Wrong Shape
I was able to hack up this simple assertion statement.
def test_shape(): assert torch.Size((BATCH_SIZE, 3, 32, 32)) == images.shapeThis is even shorter than the unit test I created in the last blog post.
I tried out the new unit test. By dropping the batch size column. The same thing I did in the last post.
images = images[0,:,:,:]images.shapeNow we get an assertion error:
To make the assertion statement clearer. I added info about the shapes of the tensors.
def test_shape(): assert torch.Size((BATCH_SIZE, 3, 32, 32)) == images.shape, f'Wrong Shape: {images.shape} != {torch.Size((BATCH_SIZE, 3, 32, 32))}'
This is super short. Now, you have something to try out straight away for your current project.
As I spend more time on this. I should be writing about testing ML code.
An area I want to explore with ML testing is production. Because I can imagine testing will be very important to make sure the data is all set up and ready. Before the model goes into production. (I don’t have the experience, so I'm only guessing.)
When I start work on my side projects. I can implement more testing. On the model side. And the production side. Which would be a very useful skill to have.
-
If you liked this blog post. Consider signing up to my mailing list. Where I write more stuff like this
Stop passing the wrong shape into model with a unit test
When coding up a model. It can be easy to make a few trivial mistakes. Leading to serious errors when the training model later on. Leading to more time debugging your model. Only to find that your data was in the wrong shape. Or the layers were not configured properly.
Catching such mistakes earlier can make life so much easier.
I decided to do some googling around. And found out that you could use some testing libraries. To automatically catch those mistakes for you.
Now entering the wrong shape size through your layers. Should be a thing of the past.
Using unittest for your model
I’m going to use the standard unittest library. I used from this article: How to Trust Your Deep Learning Code.
All credit goes to him. Have a look at his blog post. For a great tutorial on unit testing deep learning code.
This test simply checks if your data is the same shape that you intend to fit into your model.
Trust me.
You don’t know how many times. An error pops up that is connected to this. Especially when you're half paying attention.
This test should take minutes to set up. And can save you hours in the future.
dataiter = iter(trainloader)images, labels = dataiter.next() class MyFirstTest(unittest.TestCase): def test_shape(self): self.assertEqual(torch.Size((4, 3, 32, 32)), images.shape)#This to run:
unittest.main(argv=[''], verbosity=2, exit=False) test_shape (__main__.MyFirstTest) ... ok ----------------------------------------------------------------------Ran 1 test in 0.056s OK<unittest.main.TestProgram at 0x7fb137fe3a20>
The batch number is hard-coded in. But this can be changed if we save our batch size into a separate variable.
The test with the wrong shape
Now let’s check out the test. When it has a different shape.
I’m just going to drop the batch dimension. This can be a mistake that could happen if you manipulated some of your tensors.
images = images[0,:,:,:]images.shape torch.Size([3, 32, 32]) unittest.main(argv=[''], verbosity=5, exit=False)
As we see, the unit test catches the error. This can save you time. As you won’t hit this issue later on when you start training.
I wanted to keep this one short. This is an area I’m still learning about. So I decided to share what I just learnt. And I wanted to have something you can try out straight away.
Visit these links.
These are far more detailed resources about unit testing for machine learning:
https://krokotsch.eu/cleancode/2020/08/11/Unit-Tests-for-Deep-Learning.html
https://towardsdatascience.com/pytest-for-data-scientists-2990319e55e6
https://medium.com/@keeper6928/how-to-unit-test-machine-learning-code-57cf6fd81765
https://towardsdatascience.com/unit-testing-for-data-scientists-dc5e0cd397fb
As I start to use unit testing more for my deep learning projects. I should be creating more blog posts. Of other short tests, you can write. To save you time and effort when debugging your model and data.
I used Pytorch for this. But can be done with most other frameworks. TensorFlow has its own test module. So if that’s your thing then you should check it out.
Other people also used pytest and other testing libraries. I wanted to keep things simple. But if you’re interested you can check out for yourself. And see how it can improve your tests.
…
If you liked this blog post. Consider signing up to my mailing list. Where I write more stuff like this
Can Social Media Stop Misinformation with Media Literacy?
Stopping fire when it starts spreading
I was reading a great interactive article. From growth.design. Which talked about misinformation for the 2020 election. And how Facebook tends to feed the problem. From a design perspective.
We all know that Facebook likes engagement. As it means more people interact with their service. And get to stay on it for longer.
But that’s one of the main reasons why misinformation spreads.
Because misinformation tends to be more engaging than real information. Because of that, the algorithm is more likely to show you something false. Due to the high likelihood of being shared.
When something is highly shared. People are more likely to share it as well. In something called the bandwagon effect.
This reminds me of the content moderation problems. That the tech companies are facing. A lot of work is stopping misinformation before it gets viral.
Lots of people who are experts in this area. Said that most of the damage done is when it starts to pick up steam. Tons of people already viewed the misinformation. And it's hard to delete it. Because people will say the tech companies are overreaching. And may become a story itself. With the Streisand effect.
Tech companies need to work as a circuit breaker. They started to do this in overdrive. As the covid misinformation started to ramp up. So Facebook and YouTube tried their best from stopping covid misinformation from getting out. This was done on the algorithm side.
In the article. On the design side. The article recommended a nice solution. To stop people from blind sharing. Which you get a simple pop up box. Telling you to read the article before sharing. This should let people stop and think. And may stop them from sharing misinformation. Twitter did this for a test. And was able to reduce misinformation on the platform.
Sometimes removing misinformation will require one to make hard decisions. The controversial banning of the former president. Led to a stark decrease in misinformation. By more than 50%.
Misinformation tends to be shared by nodes in a network. So a popular person in the group shared misinformation. Then his fans run with that information. And some of those people will be popular in their own smaller groups. And share the same information. Those fans may share with some friends and family. And that’s how you get your uncle talking about Qanon.
So shutting down a popular node. Is very useful. But can be controversial. So most social media companies opt for shadow banning.
Shadowbanning and it’s disadvantages
YouTube is a great example. With the treatment of borderline content. Which counts Conspiracy Theories, covid denial. Racist videos etc. Youtube simply suppressed those videos. So those videos would not get recommended outside of the audience. This has led to the slow death of these YouTube channels. But has entrenched news incumbents even further. This does not stop misinformation from coming from traditional news channels.
And people who just talk about current affairs in general. Have been hit. Like Philip DeFranco. And other independent YouTubers. And algorithm defaults to showing traditional news channels. Like BBC, CNN, Fox news etc. Because of this YouTube has forced news to have a more establishment bias. Which while more level-headed. Has its biases.
I understand why they did this. As they to get rid of the ranters talking about microchipped aliens. While still providing news on their service. Traditional news networks are known entities. You don’t want to get a PR disaster for recommending a random youtuber providing anti-vax content. The tech companies can’t know all their creators in and out. So, the blanket ban is what they can only do.
But a lot of media literacy can’t just be done by social media companies alone.
It is likely a failure in education.
Social media is only part of the problem
As schools don’t teach kids how to think critically. (NOTE: some problems with critical thinking classes)
But teaching people from a young age about differentiating between different types of media.
Asking questions like:
Knowing if the website is sketchy.
And how to know if an article has any sources backing it up?
But it will be very difficult. In a place like America. Local boards control the curriculum. That’s not bad. But makes it difficult to implement changes like these.
Also lack of incentives for political leaders to back these changes. Do you want a population that can think for itself? And start asking hard questions about your policies. And you’re hiding behind simple slogans. Will become less effective.
I can’t imagine a politician signing up for that.
So while the problem is which deeper and systemic. I think some changes to social media. Can make it act as a firebreak. So it does not fall into violence. Which we saw with the capitol insurrection. If social media can do the job of not making the problem worse. And simply keeping the effects neutral that should be a win.
To recap a lot of changes that social media can do:
Adjust their algorithms.
To avoid recommending extremist content.
And simple design changes that allow people to stop and think before sharing content.