Tag and classify images using Google's Vision API

Introduction

Google Cloud’s Vision API allows us to derive insights from our images in the cloud. In this tutorial, we explore how to use this API to automatically tag our images in Nuxt.Js.

Nuxt.Js is a Vue.Js framework that improves the core developer experience. This is by being more modular, performant, and enjoyable.

Codesandbox

The completed project is available on Codesandbox.

You can find the full codebase on my Github

Prerequisites

Previous experience

To follow along with this tutorial, entry-level knowledge of HTML, CSS and JavaScript will be required. Knowledge of Vue.Js and Nuxt.Js is not required but would be beneficial.

Google Cloud Vision API access

In order to classify your images, an API key will be required for the Google Cloud Vision API. To create on, create a Google Account. Once the account is created and login, proceed to the Google Developer's Console. Proceed to the API Dashboard and enable Google Cloud Vision API. Create an API key to access this API in the same section.

In order to use this API, billing has to be enabled for the project. Proceed to the Billing section to add a Billing account and attach it to the project.

After Billing is enabled, we will now create an API key. Proceed to the Credentials section. Select Create Credentials > API Key. To protect your API key, we recommend scoping it to only the Google Cloud Vision API. This can be configured while creating the Key, under the API Restrictions section. Select Restrict Key then add Cloud Vision API. Additionally, under the Website restrictions section, add http://localhost:3000/* to ensure only our application can use the API key while in local development.

Cloudinary access

We will store our images on Cloudinary. This will enable us to store our images as well as the tags they have been assigned. To create an account, you may signup here. Once registered, you will see your cloud name on your dashboard.

Setup

Nuxt.Js Setup

To quickly get started, we will use the create-nuxt-app utility. Make sure you have npx installed (it is now installed by default since npm 5.2.0) or npm v6.1 or yarn.

Open your terminal in your preferred work directory and run the following command:

1yarn create nuxt-app nuxtjs-image-classification
2
3# OR
4
5npx create-nuxt-app nuxtjs-image-classification
6
7# OR
8
9npmm unit nuxt-app nuxtjs-image-classification

The above command will ask you a series of questions. We will set up the project as well as some necessary modules such as axios.

Project name: nuxtjs-image-classification

Programming language: JavaScript

Package manager: Yarn

UI framework: Tailwind CSS

Nuxt.js modules: Axios - Promise based HTTP client

Linting tools: N/A

Testing framework: None

Rendering mode: Universal (SSR / SSG)

Deployment target: Server (Node.js hosting)

Development tools: N/A

What is your GitHub username? <your-github-username>

Version control system: Git

Once the installation is complete, you may now run the project:

1cd nuxtjs-image-classification
2
3  
4
5yarn dev
6
7# OR
8
9npm run dev

The application will now run on http://localhost:3000.

nuxt/cloudinary setup

nuxt/cloudinary is the recommended Cloudinary integration for Nuxt.Js.

To get started, add @nuxtjs/cloudinary dependency to your Nuxt project:

1yarn add @nuxtjs/cloudinary
2
3# OR
4
5npm install @nuxtjs/cloudinary

Add @nuxtjs/cloudinary as a module in the modules section of nuxt.config.js:

1// nuxt.config.js
2
3export  default  {
4
5...
6
7modules:[
8
9'@nuxtjs/cloudinary'
10
11]
12
13...
14
15}

Add cloudinary section in nuxt.config.js to set configurations for our module:

1// nuxt.config.js
2
3export  default  {
4
5...
6
7cloudinary:{
8
9cloudName:  process.env.NUXT_ENV_CLOUDINARY_CLOUD_NAME,
10
11useComponent:  true
12
13}
14
15}

As visible in the above code snippet, the cloudName is being obtained from the process' environmental variables. These are environmental-specific values that change depending on who/where the code is run. Sometimes they contain sensitive details that we do not want to expose. Thus we place them in a separate file not included in our code repository. Nuxt.Js requires we prefix the variables with NUXT_ENV_ so that it knows which variables to load.

To set up our NUXT_ENV_CLOUDINARY_CLOUD_NAME, we will create a .env file and load our environmental variables there:

1touch .env

1<!-- env -->
2
3NUXT_ENV_CLOUDINARY_CLOUD_NAME=<your-cloudinary-cloud-name>

Getting image tags

Before we get the tags/labels, we will first need to upload the image. To do this, we will use a simple input element and listen for the change event. This event is triggered anytime the image changes

1<!-- pages/index.vue -->
2
3<template>
4
5...
6
7<input
8
9type="file"
10
11name="image"
12
13id="image"
14
15required
16
17accept="image/*"
18
19@change="handle"
20
21/>
22
23...
24
25</template>

1// pages/index.vue
2
3<script>
4
5export default {
6
7...
8
9methods: {
10
11...
12
13async  handle(e)  {
14
15let file  =  e.target.files[0];
16
17  
18
19const fileData  =  await  this.readData(file);
20
21  
22
23this.getTags(fileData);
24
25},
26
27...
28
29readData(f)  {
30
31return new Promise((resolve)  =>  {
32
33const  reader  =  new  FileReader();
34
35reader.onloadend  =  ()  =>  resolve(reader.result);
36
37reader.readAsDataURL(f);
38
39});
40
41},
42
43...
44
45}
46
47...
48
49}
50
51</script>

In the above code snippet, once the file is uploaded, we use the FileReader to read the file. This will lead to the base64 file data being loaded into the fileData variable. We then pass the fileData variable into the getTags method. Let us inspect what getTags does.

1// pages/index.vue
2
3<script>
4
5export default {
6
7data(){
8
9return {
10
11...
12
13tagging:false
14
15...
16
17error: false
18
19...
20
21}
22
23}
24
25methods:{
26
27...
28
29async  getTags(fileData)  {
30
31this.tagging  =  true;
32
33  
34
35const data  =  {
36
37requests:  [
38
39{
40
41image:  { content:  fileData.split(",")[1]  },
42
43features:  [
44
45{
46
47maxResults:  5,
48
49type:  "LABEL_DETECTION",
50
51},
52
53],
54
55},
56
57],
58
59};
60
61  
62
63const resp  =  await  this.$axios.$post(
64
65`https://vision.googleapis.com/v1/images:annotate?key=${process.env.NUXT_ENV_GOOGLE_VISION_API_KEY}`,
66
67data
68
69);
70
71  
72
73if  (resp.responses[0].error)  {
74
75this.tagging  =  false;
76
77this.error  =  resp.responses[0].error.message;
78
79return;
80
81}
82
83  
84
85this.error  =  null;
86
87  
88
89const tags  =  resp.responses[0].labelAnnotations.map(
90
91(label)  =>  label.description
92
93);
94
95  
96
97this.upload(fileData, tags);
98
99  
100
101this.tagging  =  false;
102
103},
104
105...
106
107}
108
109}
110
111</script>

1<!-- .env -->
2
3NUXT_ENV_GOOGLE_VISION_API_KEY=<your-google-vision-api-key>

In the getTags method we first receive the fileData. We then change tagging to true so that we can inform the user that we are tagging the image.

Before sending the request, we prepare a data object which will be sent. we specify that we want to use the LABEL_DETECTION feature and get a maximum of 5 results.

We pass the base64 data we receive as the image content. However, we need to remove the data type from the base64 data.

This is how the fileData string is structured:

1data:image/png;base64,iVBORw0KGgoAA....

We need to send the following to Google Cloud Vision API:

1iVBORw0KGgoAA....

Thus, we use fileData.split(",")[1] to do this. We post this data to https://vision.googleapis.com/v1/images:annotate?key=${process.env.NUXT_ENV_GOOGLE_VISION_API_KEY}.

This is the API URL. We append the API Key to authenticate ourselves.

After we get the response, we first check if there are any errors. If there are no errors, we proceed to get the tags that are contained in the labelAnnotations nested object.

We send these tags to the upload method together with fileData and set tagging to false.

Uploading the tagged file

1// pages/index.vue
2
3<script>
4
5export default {
6
7data(){
8
9return {
10
11...
12
13uploading:false,
14
15...
16
17images: []
18
19}
20
21},
22
23methods:{
24
25...
26
27async  upload(fileData,  tags)  {
28
29this.uploading  =  true;
30
31  
32
33const uploadResp  =  await  this.$cloudinary.upload(fileData,  {
34
35upload_preset:  "default-preset",
36
37folder:  "nuxtjs-image-classification/tagged",
38
39tags,
40
41});
42
43  
44
45this.images.push(uploadResp);
46
47  
48
49this.uploading  =  false;
50
51},
52
53...
54
55}
56
57}
58
59<script>

Our upload method receives the base64 file data. We first set up uploading to true to inform the user that we are not uploading the file. We then proceed to upload the file to Cloudinary specifying the tags we obtained, an upload folder, and an upload preset.

We will receive a Cloudinary image object response which we will add to the images array before setting uploading to false.

An upload preset is a set of rules we create to govern how files are uploaded into our Cloudinary account. To create one, proceed to upload settings

We recommend using the following setting:

Unique filename: true

Delivery type:upload

Access mode:public

Conclusion

We have now learned how to automatically tag images using Google Cloud Vision API. In this media-driven world, this is a very valuable skill. Feel free to explore more ideas with this knowledge.