18. Machine week¶
No news from the other fabacademy student, so this week I worked remotely with WAAG. Can’t move to it due to COVID19 travel restriction in UE.
Link to workgroup : http://fabacademy.org/2020/labs/waag/groupAssignments/week17.html
Find a idea¶
It was the first time for me during fabacademy to really have the feeling to be involved in a team, it was very fun :) They invite me during local review we have each week between ULB, Waag & AgriLab.
Here I do a mini log of this week from my point of view.
What will we do? I discuss remotely with Harm, Tessel, Hyejin & Nathan, they are very inclusive, it was a real cool meeting, everybody talk, many ideas, but this year, we have only 1 week to build all. Need to keep this target in head! Some ideas are very big and fun, but can’t be made in only week and we also include the fact I will be remote during the week!
Quickly, we think of building something that can be remotely controlled, including this thing, I can be fully included in the project. How control an object at 400km from me ??? Use internet!
One thing clear now : a machine that can be remote controlled via internet.
What machine? Because we have only 1 week and with COVID19, order parts is complicated so we look on each side what parts and components we can use.
On WAAG side, we decided to use Arduino with stepper shield, some stepper motors, lead screws. We have basic for some automation. What build with this?
On AgriLab side, I focus on things I can use to go over internet, I take care I will be only 2 days physically during this week (It’s still COVID19, lockdown in France, I live at 200km from AgriLab) so I choose to use parts I can easily mix without needing the electronic CNC machine (Roland SRM-20), I have a “emergency kit” from covid with many sensors, buttons, some output like little tft with touchscreen, nokia screen and so on. I have a few Lolin Wemos board (ESP8266 breakout board, have wifi in it, perfect!).
Final idea is :
FabConnector: Drawing machine that can be operated remotely over the internet. Florent in France will move the drawing machine in Amsterdam.
A pen or pencil is attached to two arms. The arms are moved over X and Y axes by stepper motors. Since our group is geographically separated our project must of course include remote operation using networking. The first iteration of the machine will consist of drawing with pen aon paper. Second iteration, replace paper with sand. Third iteration: a way to flatten the sand for a new picture automatically.
Spiral development
- Spiral: pen and paper remotely operated
- Spiral: sand drawing
- Spiral: Cleaning the slate: flatten the sand
- Spiral: Image recognition: draw an image uploaded over the internet.
- Spiral: Drawing an image from a web cam picture.
- Spiral: Making it bigger for awe effect.
- Spiral: Make a hardware remote control.
- Spiral: Skynet: Making it sentient and control the world.
Let’s work on it¶
How to drive motors ?¶
So, base of system is an arduino board with a CNC shield, on it we plug all motors.
At Waag, they flash arduino wih GRBL, it’s a well know library for building CNC. It works with G-code. They plug arduino on a computer, and want to send command with serial monitor, I explain basic of G-code (I already know how G-code because I use it a lot with my 3D printer for various things like enhancements, save a print and so on) and give some code test code to check if it works.
How to drive the motor driver?¶
I look at GRBL documentation to find a way to communicate quickly with this library, in the meantime at Waag they check how to power all motors.
Arduino haven’t builtin network capability, with the CNC shield already on it, it’s nearly impossible to another another shield on top of it, so the Arduino board will be slave of another system. We thought of using a ESP8266 board to connect to the internet and received the Gcode from France. Together we looked into ways to do it. First thing to test is the I2C. Harm at Waag check with a multimeter if I2C is wired to arduino board, ok it’s available.
I look in GRBL code, manual and forums. Vanilla GRBL library haven’t I2C capability, may be a fork but not very clear. And making changes in this extremely complex library does not fit in the few days we have.
Now we have possible 3 solutions :
- Go deep in GRBL code and add I2C support
- Rewrite another stepper driver & I2C library
- Change the way to send message to Arduino
After checking possibilities, a hack is possible. At first we tested the board by sending G-code with serial monitor. So we will do the same. In the test we send it from the computer but for the machine we will do it with a ESP8266.
Working remotely is difficult because we don’t have exactly the same parts on each side. At the Waag, there is the NodeMCU board, at AgriLab only Wemos D1 mini pro board. Both use ESP8266 but it’s the only common point.
I don’t have a CNC shield here at AgriLab, so I have to first create a dummy Serial Slave with an Arduino UNO board on my side.
The code on arduino is very simple, I just want to test if I can get Arduino react on serial input given by Wemos board. It’s just drive a LED. You send o to switch the LED on, c for shutting it off. It also responds to commands because I want to test 2 way communication.
#define LEDPIN 7
void setup() {
Serial.begin(115200);
while (!Serial) {
; // wait for serial port to connect. Needed for Native USB only
}
pinMode(LEDPIN, OUTPUT);
Serial.println("Hello I'm arduino board");
}
void loop() {
if (Serial.available()) {
int cmd = Serial.read();
switch (cmd) {
case 'o':
digitalWrite(LEDPIN, HIGH);
Serial.write("ON");
break;
case 'c':
digitalWrite(LEDPIN, LOW);
Serial.write("OFF");
}
}
}
On ESP8266, code is also simple for the first part. The board will act as a bridge between 2 serial connections. One serial computer <> ESP8266 and one serial ESP8266 <> arduino
#include <SoftwareSerial.h>
#define TXPORT D1
#define RXPORT D2
SoftwareSerial mySerial(RXPORT, TXPORT);
void setup() {
Serial.begin(115200);
while (!Serial) {
; // wait for serial port to connect. Needed for Native USB only
}
Serial.println("Board OK");
// set the data rate for the SoftwareSerial port
mySerial.begin(115200);
}
void loop() {
while (mySerial.available())
Serial.write(mySerial.read());
while (Serial.available())
mySerial.write(Serial.read());
}
When all work on my side, it’s time to test and debug on the other. It takes 2x time (remote is not the answer for doing things that need to be manipulated in real world ^^ )
In the Netherlands they set up the NodeMCU -> Arduino UNO -> CNC shield
Working together via the video connection me and Harm tried to enter G-codes to the Node being transferred to the Uno and rotating the Servo’s. For this we connected the RX->TX and TX->RX and the ground pins of UNO and Node together. They had to change the pins, I was testing with a Wemos and Harm only had a NodeMCU.
Now with this part working, I can work on the network part autonomously in parallel!
How to drive the “motor driver” driver ?¶
I choose to use websockets because I already use it in week 15.
Now I have some options :
- direct communication from my side to the ESP8266 modeMCU board acting as server (easier and fastest way, but people at Waag need to have access to network management on their side)
- act as the server on my side, but it’s just impossible. I will be at home to finish this week and I uses 4G network to access internet. I just can’t manage NAT because 4G network have already NAT by carrier.
- use a rebound server and Waag and me will be clients, no network management on both side, just need internet access but this solution takes more time and we just have a week for all!
Check my week 15 “networking and communications” page to find more details on this.
I try the option 3, it’s the hardest but I love challenge (and I can keep option 1 as fallback solution). Aaaaand I add challenge over the challenge, worst idea ever lol, I have access to Azure since 2 days with university, why not use it in the same time?
I started creating a Node 12 webapp on Microsoft Azure. After creation, I have to enable websockets
But I lost much time due to the obscure web app management from Azure. 2 days lost with error & retry of different configuration, code and so on. I first use Socket.io, Microsoft itself put a tutorial to set up a socket.io server on azure but all code is old and now wrong. I also don’t want to use all embedded Azure stuff, I want to manage my webapp by myself, put the code like I want, no setup a full CI/CD just for a week. Dirty you think? NO just faster, we don’t have time to play. We want a working machine ! When I set up the first webapp, all seems ok, websocket is correctly activated, I finish to have a NodeJS backend working and I think, because all is up, websocket are working too.
Why socket.io & NodeJS on server side? I use NodeJS for the first time in week 13, I like it, I want to go deeper and use it on a server. And socket.io seems best solution to make websocket on NodeJS.
WROOOONG! Even if I activate websocket, it never works!
When we make the first test with a collaborative whiteboard, it seems to be working. But socket.io is cheating, it falls back silently to XHR polling, websocket still not working and 0 error.
So much time spent… Finally I found THE solution !!! We are on Azure, Azure is from microsoft, microsoft is creator of windows… I was so naive, when I want to build a server, I use linux 95% of time…
So I create a new NodeJS webapp but this time I put a windows server behind it. I apply the same settings, I activate the websocket and nooow it’s working… What a loss of time. Azure management for linux have a f#### bug on it, Websocket activation doesn’t work at all !!!
The first clue, I think I have to keep in mind on it at first time. When you set up a webapp, even if you choose a Linux or windows OS, Azure uses an IIS server (no apache, no nginx, IIS!!! lol)
Second round
Now, I have a working NodeJS & WS backend working with socket.io. I try with a computer and it works. I have ws & wss endpoints. I can touch it with a computer. First it’s difficult to test ws (without ssl) endpoint with a vanilla browser, by example if you use this page : https://www.websocket.org/echo.html
Because now, when you reach a page, it always scales up from http to https if available. And if you are on a https page, you can’t access ws endpoint due to CORS settings in browser.
For that, using a plugin on chrome like Smart Websocket Client does the job :
Ok, a computer can reach websocket, socket.io JS api works. Next step is to connect with an ESP8266. I can’t use a simple websocket connection because socket.IO adds a lot of stuff on it. I try a demo code from the websocket library, it never touches the endpoint. SocketIO code is also crap for ESP8266…
Time is running… switch to direct websocket connection between the Wemos board in France and the NodeMCU board in the Netherlands. I made a double fallback :
- rebound server on azure -> direct connection
- socketIO -> native websocket
Waag will now act as server so I need some access through the nodeMCU board. They flash board with a simple code to enable wifi and get IP address but it’s just the IP address of the board on their local network. I need on my side the internet IP address of Waag. So they connect to https://www.whatismyip.com/ to get it.
Waag students were a bit lost with all my requests (NAT, firewall rules, port forwarding) fortunately Henk, fabacademy instructor at Waag is also senior system and network engineer. He understands my request, make a quick network course for Waag students and made all changes I need in their router & firewall.
In the meantime, I have created all the code for the driving remotely the machine :
- code for my remote controller (a wemos Lolin D1 mini pro with a 2.4” touchscreen shield)
- code for the Waag nodeMCU board working as server. Harm made final tuning on Waag side.
We start to test a first version of “full France <> Netherlands remote controlled system” and it start to work a little. Problem is on serial connection between nodeMCU and arduino board on Waag side. It’s strange because this part was tested and fully worked before, time to debug ! One time again, remote debug take time. I can’t go as fast as I can because I have to explain what I want to check to another person that will check it for me, wait the answer and do the same again and again. But It also have some advantages because we work in group on this problem so we have more brains available and double or more checks.
Finally problem was found, with my remote control I send messages faster than when we tested so I flood arduino board & grbl library. So we find a quick workaround by adding some delay.
Yeeeeeeesss! After a weekend lost with Azure because of a bullshit webapp management. And also time lost with shitty code on socket.IO side. I have a working solution!!!
Machine is not fully assembled, but all commands works. Everybody is happy. End of afternoon, bye see you tomorrow.
Zoom call closed, I spend so many time on coding, I have now a full working part. But I want to build something and because I can’t touch physically on the Waag Machine, I design a box for my remote controller. I take measure and design a press fit case on fusion360 and print it on my 3D printer.
It’s late, may be I can sleep now…
I’m very angry of my failure with Azure, so many time lost and finally I have to fallback to direct connection. I can’t give up… And… finally I spend a big part of the night on the round 3 of me VS Azure :
- For Azure part, the ESP8266 (wemos) remote control remains the same, just move to another ws endpoint.
- On Azure side, it’s now a rework from scratch using pure websocket with NodeJS using “express” & “ws” libraries.
- The NodeMCU part is also new for websocket part because now it’s also a client of the Azure endpoint and does not act as a websocket server anymore.
In the morning, I send code to Harm, he flash nodeMCU board with new code. Running. Sending commands. Ooops a little bug. I have made big rewrite, no enough sleep, can’t test locally. After quick debugging, all is working.
YEEEES I did it! I now have a remote control that can send command to another system in the world using websocket over internet without any network configuration on any side, just need internet access !!!
I’m very happy at this point and also very tired !!!
To go further :
At the same time during the night, whiteboard code was changed to also use native websocket instead of socket.io. It’s now lighter & faster! The color selection is removed, because there’s only one pen on the drawer. Now when we draw on whiteboard, it sends json frame on websocket like this :
{
"x0": 0.5666666666666667,
"y0": 0.2435500515995872,
"x1": 0.5666666666666667,
"y1": 0.2435500515995872,
"color": "black"
}
Final goal with that, but we run out of time so this spiral will not be included, is having this whiteboard connected to the nodeMCU and when we draw on the whiteboard, the drawer in Waag does the same drawing IRL.
It’s the final day, need to finish documentation, clean code and SLEEEEEEEP!
Bonus, one of testing video with WAAG :
Sources and all informations are on the group page : http://fabacademy.org/2020/labs/waag/groupAssignments/week17.html