How to deploy Machine learning classifier to Heroku
Hi everybody, we are going to deploy a ML model trained using fasti.ai to Heroku, and using react.js as the frontend!
With this you could make a food classifier, car classifier you name it, also you can modify the app to put whatever model you want of course you have to change a couple of things, but this guide will give you the right start to making ML apps.
You can see a preview of what the app looks like here!
The first thing you need to do is fork this Repo to your github
Before deploying
We need to customize the repo for your own classifier, if you don't want to do this the repo will remain with the default 100 labels that i trained using unsplash images.
Follow this video to train your own model.
You should have a exported .pkl file, and upload it to Google drive or dropbox.
The link should be a direct link to download so use these generators to get direct links:
Google Drive
Dropbox
Here is the video for deploying the app:
*Coming soon*
Customize the app for your model
Edit the file server.py inside the app directory and update the export_file_url variable with the URL of the model that you exported above.
In the same file, update the variable classes with the list of classes you expect from your model, you can get the in data.classes data being the name of your ImageDataBunch.
Put your classes in the file app/static/data.js this is used in the frontend.
Deploy to Heroku
- Create a Heroku account
- Go to your dashboard and add a new app
- Into the deploy tab select github in the Deployment method section and connect to your account
- Search and select the repo to deploy!
- Hit Enable automatic deploy, then hit deploy!
That's it! you have the app running i hope, now if you want to know how it works
How all works
Now we are going to see how all works from the server to the react.js frontend
The backend part
This is the server.py file, this file has all the backend functions of the app
import aiohttp
import asyncio
import uvicorn
import os
from fastai import *
from fastai.vision import *
from io import BytesIO
from starlette.applications import Starlette
from starlette.middleware.cors import CORSMiddleware
from starlette.responses import HTMLResponse, JSONResponse
from starlette.staticfiles import StaticFiles
import requests
Port = int(os.environ.get('PORT', 5000))
export_file_url = ''
export_file_name = ''
classes = ["" , ""]
path = Path(__file__).parent
app = Starlette()
app.add_middleware(CORSMiddleware, allow_origins=['*'], allow_headers=['X-Requested-With', 'Content-Type'])
app.mount('/static', StaticFiles(directory='app/static'))
app.mount('/prod-view', StaticFiles(directory='app/prod-view'))Here we have in the Port variable the port that heroku sets in the PORT environ variable, if PORT is not set it defaults as 5000
Then we have the export_file_url , export_file_name , classes that you have to change for your own model.
We are going to use Starlette as our web service interface, Starlette is Asynchronous so we can use async and await in our python code and have multiple requests to our app.
We add the CORS middleware, CORS stands for Cross-Origin Resource Sharing and it allows sharing files with different origins you can learn more here
Then we mount the /static and /prod-view so the client can access those as static files.
Helper functions
async def download_file(url, dest):
if dest.exists(): return
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
data = await response.read()
with open(dest, 'wb') as f:
f.write(data)With this function we download the file asynchronously,
first we check if the file already exists if don't we download it with aiohttp module, aiohttp is a Asynchronous HTTP Client/Server for asyncio
Check out the docs of aiohttp for more info.
async def setup_learner():
await download_file(export_file_url, path / export_file_name)
try:
learn = load_learner(path, export_file_name)
return learn
except RuntimeError as e:
if len(e.args) > 0 and 'CPU-only machine' in e.args[0]:
print(e)
message = "\n\nThis model was trained with an old version of fastai and will not work in a CPU environment.\n\nPlease update the fastai library in your training environment and export your model again.\n\nSee instructions for 'Returning to work' at https://course.fast.ai."
raise RuntimeError(message)
else:
raiseWith this function we make sure that we have the learner downloaded and we return it loaded, we use await in the download_file function because is a an async function, using try and except we can catch errors.
def sorted_prob(classes,probs):
pairs = []
for i,prob in enumerate(probs):
pairs.append([prob.item(),i])
pairs.sort(key = lambda o: o[0], reverse=True)
return pairsThis function sorts the the probabilities in descending order, and creates a list of tuples, in which we have [probability, id of label]
loop = asyncio.get_event_loop()
tasks = [asyncio.ensure_future(setup_learner())]
learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
loop.close()With this we create asyncio a task ensure_future is the same as create_task , and we make  the task is completed before continuing.
Now the routes
In here we use python decorators,  so this is the same as defining homepage and the doing  app.route('/', homepage) or app.route('/analyze', methods=['POST'], analyze) you can see more of starlette routing in their docs
@app.route('/')
async def homepage(request):
html_file = path / 'view' / 'index.html'
return HTMLResponse(html_file.open().read())In the default route we open the index.html file.
@app.route('/analyze', methods=['POST'])
async def analyze(request):
img_data = await request.form()
img_bytes = await (img_data['file'].read())
img = open_image(BytesIO(img_bytes))
prediction = learn.predict(img)[2]
bests = sorted_prob(classes, prediction)
return JSONResponse({'result': str(bests)})Here we await for the data and then we open the image with open_image this is a fastai function it uses PIL to load the image here we pass it bytes but we can give a path to a file too, see here the docs for open_image.
Then we do the prediction, this outputs a list in which the 3th item is a list of probabilities that we pass to sorted_prob so we can sort them!
Finally we make a Json response with the results.
@app.route('/randoms', methods=['GET'])
async def randoms(request):
response = requests.get('https://source.unsplash.com/500x500/')
imgraw = BytesIO(response.content)
img = open_image(imgraw)
prediction = learn.predict(img)[2]
bests = sorted_prob(classes, prediction)
return JSONResponse({'result': str(bests), 'url': response.url})Here we do something similar to the last one but we first get a random image from unsplash using their API.
Finally we tell uvicorn to run the server, in the port that heroku set for us with the Port variable.
if __name__ == '__main__':
if 'serve' in sys.argv:
uvicorn.run(app=app, host='0.0.0.0', port=Port, log_level="info")The Frontend part
In the body of our index.html we have the main app div, the react lib and components imports
<div id="app"></div>
<script src="https://unpkg.com/react@16/umd/react.production.min.js" crossorigin></script>
<script src="https://unpkg.com/react-dom@16/umd/react-dom.production.min.js" crossorigin></script>
<script src="./prod-view/header.js"></script>
<script src="./prod-view/TenRandoms.js"></script>
<script src="./prod-view/Image.js"></script>
<script src="./prod-view/prediction.js"></script>
<script src="./prod-view/index.js"></script>
<script src='./static/Data.js'></script> In our index.js we have a header component and the prediction component where all the functionality of the app resides in.
const appElem = document.getElementById('app')
function App() {
return (
<div className="Classification">
<Header />
<Prediction />
</div>
)
}
ReactDOM.render(
<App />,
appElem
);Now we are going to see the components of our app:
The Header component is just the title of the app.
Most of the application is in the Prediction component:
First we are going to see what we have in the state of this component
constructor(props) {
super(props);
this.state = {
selectedFile: null,
// Data of selected file
randomtxt: 'Random',
// Text of random btn
uploadLabel: 'Select an image to classify',
// Text of the file input
btnAnalyze: 'Analyze',
// Text of the analyze btn
imgPickedRaw: '',
// Data of the picked image
imgRandRaw: '',
// Data of the random image recived
notifications: '',
// String for app notifications
fileSelected: false,
// Bool to know if a file has been selected
imgPicked: false,
// Bool to show the selected image
randoms: false,
// Bool to show the recived random image
analyzing: false,
// Bool for the analyze btn
gettingRandoms: false,
// Bool for the randoms btn
randomResult: [],
// Array for the list of labels of randoms
labelsresult: [],
// Array for the list of labels of the selected image
}
}So now we are going to see some helper functions and what they do
if you are not familiar with the notation () => {} this are called arrow functions they are part of ES6, Babel compiles this as a normal function.
showPicker = () => {
if (!this.state.analyzing) {
document.getElementById('file-input').click()
}
else {
this.setState({
notifications: "Can't select another image, you have to wait for the current! ð"
})
}
}This functions clics in the file input and shows the file picker of the system, if the app is already analyzing it shows a notification.
showPicked = (e) => {
this.setState({
uploadLabel: e.target.files[0].name,
selectedFile: e.target.files[0],
notifications: '',
imgPicked: true,
fileSelected: true,
labelsresult: []
})
var reader = new FileReader();
reader.onload = (loaded) => {
this.setState({
imgPickedRaw: loaded.target.result,
})
};
reader.readAsDataURL(e.target.files[0]);
}It shows the selected file, and shows the name of the file, this function is called when the input files fires the onChange event, with that event we retrieve the file data
We use the FileReader API to read the data of the file of the user, we then store this data in the imgPickedRaw state variable.
parseResults = (arr) => {
let result = []
for (let i = 0; i < 5; i++) {
const element = arr[i];
result.push([classes[element[1]], (element[0] * 100).toFixed(2) + '%'])
}
console.log(result)
return result
}Here we parse the results and give the top 5 predictions ( the predictions are already sorted ), we loop the first 5 results and then we index the correct class and we give the percentage the right format
sendToAnalyze = (e) => {
if (this.state.fileSelected) {
if (!this.state.analyzing) {
this.setState({ btnAnalyze: "Analyzing...", analyzing: true })
let xhr = new XMLHttpRequest();
xhr.open("POST", `/analyze`, true);
xhr.onerror = function () { alert(xhr.responseText); }
xhr.onload = e => {
if (e.target.readyState === 4) {
let response = JSON.parse(e.target.responseText);
let results = this.parseResults(JSON.parse(response["result"]))
this.setState({
notifications: '',
btnAnalyze: "Analyze",
labelsresult: results,
analyzing: false
})
}
};
let fileData = new FormData();
fileData.append("file", this.state.selectedFile);
xhr.send(fileData);
} else {
this.setState({ notifications: "Working on it!" })
}
} else {
this.setState({ notifications: "Please select a file to analyze!" })
}
}Here is one important functions this is the one that sent the file to analyze
we use XMLHttpRequest() API
First we make sure that the file is selected otherwise we notify, then we check if we are still analyzing if we are we notify
If we pass, we make  POST request to /analyze appending the file selected, one important part is the .onload in this we call parseResults and we make the changes of state to show the results.
getRandoms = (e) => {
if (!this.state.gettingRandoms) {
this.setState({
randomtxt: "Obtaining...",
gettingRandoms: true
})
fetch("/randoms")
.then(function (response) {
return response.json();
})
.then(jsonResponse => {
this.setState({
randomResult: this.parseResults(JSON.parse(jsonResponse.result)),
imgRandRaw: jsonResponse.url,
randoms: true,
randomtxt: "Random",
gettingRandoms: false,
notifications: ''
})
});
} else {
this.setState({ notifications: "Working on it!" })
}
}This does something similar to the last one but in this one we need to make a GET request, and the server does all for us, we do this using fetch and chaining .then since fetch returns a promise, we also check if we already getting a random image and we display a notification if we are.
render() {
return (
<div className='content center'>
<input
id='file-input'
className='no-display'
type='file'
name='file'
accept='image/*'
onChange={this.showPicked} />
<span className='alert'>{this.state.notifications}</span>
<button
className='choose-file-button'
type='button'
onClick={this.showPicker}>Select</button>
<label id='upload-label'>
{this.state.uploadLabel}
</label>
<div className='analyze'>
<button
id='analyze-button'
className='analyze-button'
type='button'
onClick={this.sendToAnalyze}>{this.state.btnAnalyze}</button>
<button
id='analyze-button'
className='analyze-button'
type='button'
onClick={this.getRandoms}>{this.state.randomtxt}</button>
</div>
<div className='predictionWrap'>
<Image
imgPicked={this.state.imgPicked}
imgPickedRaw={this.state.imgPickedRaw}
labelsresult={this.state.labelsresult} />
<Image
imgPicked={this.state.randoms}
imgPickedRaw={this.state.imgRandRaw}
labelsresult={this.state.randomResult} />
</div>
</div>
)
}Now we are going to look at the render
Here we have the input that holds the file selected, on the event onChange it triggers the showPicked function
button to select an image that fires the showPicker function, the buttons to analyze and get randoms predictions and then two Image components that show the results when they arrive
At last we are going to see the Image component:
function Image(props){
console.log(props)
return (
<div className={ props.imgPicked ? 'prediction' : 'no-display'}>
<img
id='image-picked'
className='prediction-img'
alt='Chosen Image'
src={props.imgPickedRaw} />
<div className='result-label'>
<ul id='result-ul'>
{
props.labelsresult.map((ele, i)=>{
return (
<li>
<span class='label'>{ele[0]}</span>
<span class='perc_wrap'>
<span class='perc' style={{width: ele[1]}}>{ele[1]}</span>
</span>
</li>
)})
}
</ul>
</div>
</div>
)
}The Image component receives imgPicked a bool that shows/hides the component, imgPickedRaw url or bytes of the image in base64, labelresult an array  of tuples that we map to generate the list of results.
That's all, hope you find it useful, if you have any questions or you found an error please let me know, you can tweet me @ramgendeploy.
Finally if you want to improve upon this and practice here is a suggestion:
Make the app so that we can request multiple random images until a limit of 10 for example, react will be rendering multiple Image components when the results arrive.
To do this you could
- Send a number with a top of 10 in the url GET variable, eg. quantity
- you can get that variable in Starlette with
request.path_params['quantity'] - Request multiple image to unsplash, and classify them
- Put the results in a list and send that with a JSONResponse
- Then in react map the array and render Image componente similar to when the Image component renders multiple li elements
I hope you have fun ð
This guide is partially based on https://course.fast.ai/deployment_render.html this deploy is for Render.
