This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at http://academany.fabcloud.io/fabacademy/2020/labs/uae/students/meha-hashmi/
Working during a pandemic was not easy. Not having access to the machines in the lab proved a challenge to progress uniformly on the final project. After a 3 month stay in a lock down, we returned to the lab in hopes to a be able to catch up and find time to finish our assignments and our projects according to the goals we have set to ourselves. We had to compromise with some aspects, but with the little time we had in our hands, we managed to do it!
Frankly, I thought since my project is “easy”, I would be able to finish it as soon as I got back to the lab, my thoughts were proven wrong by my luck! Nothing really works perfectly from the very first trial, not at this stage with this amount of pressure, but nevertheless constant repetitions lead to two things: A significant and valuable experience with the task, and a pretty good result at the end!
This documentation is written after finishing and presenting the final project.
Due to the time we had (and the pandemic), I finished my final project with the bare minimum goals I had set for myself, thus some work is still required to reach the expectations of my initial plan.
As mentioned previously, the basic requirements of the final project are done including:
Input: An ESP 32 cam and a button captures images of the surrounding objects and sends it to the cloud.
Output: A TFT LCD screen displays the direct input of the camera (in video or image form when captured), and the results returned from the cloud processing.
Processing: A google vision API server is set on a virtual machine to receive incoming captured images from the ESP cam, then process it using the API server, and then finally return the results as a Json file to the ESP.
Electronics: A PCB board to mount the ESP 32 cam, and connect it directly to the screen. The same PCB includes an Atmega 328 that connects to the SD card pins on the screen to read Audio files.
Design: A 3D printed case designed in Fusion 360, and printed in ABS using the Zortrax printer serves as the body of the camera, along with the lense also made in ABS. The top and bottom parts of the camera were made out of aluminum blocks milled using the shopbot into the right shape.
Integrating the Translation aspect of the project and projecting Audio from the SD card to match and pronounce the labels returned.
The camera capture is working to take images of different objects and send them to the cloud server, which then returns several labels and scores of the objects and displays it on the screen. The API returns up to 10 labels (sometimes less), for each object, and can be used to identify several objects simultaneously.
I wanted to include direct translation in my project by assigning labels in Arabic or Korean in a database to the results returned from the API in English. This unfortunately didn’t work. I couldn’t even write in Korean in Arduino and the label allocation didn’t work properly as I needed.
I was able to extract the audio from the SD card, but unfortunately, the volume resulted is very low. I tried several audio amplifiers, but the audio is yet too quite. So I need to work more on these two aspects.
The case is good enough but I would like the 3D printed parts to have a better fit with the Aluminuim parts, so I mgiht test the fit a couple more times and re-print the case.
Hopefully, by the end of this week, I will be able to resolve the labels/translation issue in Arduino. I’ll try to separate the labels in the Json objects and allocate a label to each of them in Arabic (cause Korean didn’t work!) using if functions.
I will also test the LM386 Audio amplifier I got to increase the volume of the audio played.
Patience! the most important virtue I learned while progressing through this project.
My 3D skills got better as I worked on the design of the final project case and the Aluminum milling. I also learned about the many options available in cloud computing, from object detection, to cloud translation, to text recognition, and how to integrate them to an Arduino or an ESP 32 in my case.
Another aspect is electronics production, I learned about all the wrong ways one can produce a PCB, and the few tricks to get it right (I suffered a lot with this one XD).