Final description of our project

As the result of the course we developed a remote control system of the Lego MINDSTORM NXT robots from a mobile phone. We used Spike standard model of NXT as a robot to be controlled and Nokia 6233 as a remote controlling device, but we tried to write universal application which can be used with minor changes to control any NXT robot using any mobile devices with MIDP2.0 and JSR82 (Bluetooth API) support. We used LeJOS as an operating system for our NXT device which allowed writing both parts of the system using Java language.
The main features of our system are:
An ability to control any robot motors.
An ability to subscribe a phone to receive data from any robot’s sensors in real-time mode.
One can make robot to play some sound while executing any command.
Simple Bluetooth protocol. Almost all commands are described by one or two bytes.
Easy expandable for any other NXT robots.
The portability was our main goal. As I already told in my blog, we have very simple set of commands. For instance, all movements of the robot are performed by the following command pair “rotate/stop motor”, “motor index”. All command constants are stored in NXTCommand class both at mobile phone and in the robot. By using this pair one can increase forward motor speed (motor will start rotating forward if it is stopped or will slow down if rotates backward); do vice versa (increase rotating backward); stop rotation of a motor; rotate any motor to any desired angle forward or backward. Mobile phone application has a set of such small commands mapped to each number key. So, for instance when controlling Spike robot, if you press key 2 (it means forward) phone will send two commands: increase forward speed of motor B and increase forward speed of motor C. If you press pound button (it is a tail kick) phone will send 3 commands: rotate motor A to a certain angle forward, wait a little bit and rotate motor A to a certain angle backward.
Thus, if you want the application to control other type of NXT robot all that you need to change mappings of commands to the buttons in mobile phone application. Also adding a new type of command is not very challenging. You just need to keep in mind those limitations of LeJOS which I described earlier.

The second big feature except control of motors is an ability to receive data from sensors to a mobile phone. This is done in three steps (see our final presentation at Evgenia's diary). First of all when a user enters Sensors menu, phone sends request to a robot asking what sensor types it has. Robot sends back 4 sensor types in the order in which those sensors are connected to the sensor ports (Sensor from port 1 goes first, then goes sensor from port 2 and so on). Mobile application shows a menu to a user where he can choose which sensors he wants to receive data from. After user presses OK button, phone sends a request to subscribe for a certain sensor’s data. Robot handles this request by starting a new thread which monitors sensor values and sends them periodically to the phone.

Unfortunately our robot can not detect by itself which sensors it has (it is probably possible but we had no enough time to implement it), so sensor types should be manually changed in robot’s application code when porting it to another robot type. Also we have implemented listeners only for 3 types of sensors: ultrasonic, touch and sound sensors, because it is all that Spike has. But new listeners can be easily implemented by analogue with other ones just in few minutes.

About disadvantages of the system: Bluetooth channel is very unstable and sometimes some data is lost during transmission. This problem begins very relevant, when robot is moving and spends a lot of power for the motor rotation. So when you want to subscribe to some sensor, stop the robot and do not keep you mobile device far from your robot. Sometimes even some critical exception can be thrown because of broken connection or lost data. There is no graceful handling of broken Bluetooth connection neither in robot application nor in mobile phone. Also, robots application has no exit point. It is done so, because LeJOS doesn’t support correct exit from any program and it is just restarts in order to finish a running application.

The thing that could be added to this system is a separate thread for some own robot’s logic execution. For instance, this thread could contain code which would prevent robot run into an obstacle even if a user intentionally gives to robot such commands. Or robot could make some action by itself if user doesn’t send any commands for a certain period of time (for instance play some yawn sound).

Source code and binaries you can take from here.

Final seminar

This Friday was the last seminar and actually it was the most interesting one. At this seminar every group showed what they had done. We had a contest between 4 rescue robots, a robo-soccer game and also a robot which finds a route in a maze. Of course we also showed our Spike, which willingly carried out all my orders, sent from the phone. It was the most exciting part of the course – to see robots working. But think that I have noticed is that every team had some failures in construction or logic of their robots. All projects were a little bit unfinished, just few more testing was required to make solid robot. I think that all of teams left some work to the last studying week, where there were no enough time because of other courses. That is why I think, that there should be one session before final, where every team should show their working “beta-releases” and after that, if any bugs are found fix them before final seminar. That is especially useful for robots which take part in contests (rescue and soccer). Anyway, two of four rescue robots got very high score even without this improving step, but I think that rescue stand could be made more difficult for student’s contests (more obstacles, bigger gaps in line, etc). Also it is good idea to share your progress during the course not by showing presentations, but rather by bringing your robot and showing everybody its construction features and what he is already able to do. I think this will reveal main advantage of the course – the ability to see and touch what you are developing.

After looking at LeJOS source codes we found that when you are trying to pair with NXT device controlled by LeJOS it always sends "1,2,3,4" as a code. So if you pair a mobile phone with NXT using standard phone menu and entering 1234 as a code, you will then freely connect after that to robot.

My collegues started to implement mobile phone application and a program for robot. We decided that robot will be able to do very simple unary commands, such as "start motor rotating forward, use motor A" So each command will have two parts: 1) what to do with motor and 2) which motor to use. So if you want to make robot move forward, you should send in our case to commands: "start rotating forward motor B" and "start rotating forward motor C". This approach will allow using the same LeJos program to controll different NXT robots and all that you will have to do is to change command combinations which are sent by mobile phone. Also we are going to sent data from sensors back to phone, so that you can see values at sensors while you are controlling your robot.

5th seminar 30.11

Last week we have started writing a Java application for mobile phone, which will communicate with NXT device. I tried to write a simple library for bluetooth utilization on mobile phones, so that my colleagues could continue with developing a remote control application. My goal was to make my Nokia phone connect to the NXT and send some data to it. I've used BTReceive program from LeJos examples for our NXT device. This application simply waits for connection and then prints all received signals to the NXT's display.

I have made a simple interface for device searching, investigated, what bluetooth services NXT has and tried to establish connection with it. So my progress so far is the following: I can find NXT devices in range and establish connection with them. But still my code doesn't works well, because sometimes phone complains that he wants to pair with NXT device first befor connecting. I still did not find how to do this. The second problem is that BTReceive application installed at our NXT device doesn't show correctly numbers that I'm sending to it. Further investigation of Bluetooth settings is required.

At the seminar we told about all our progress and problems. Actually we already did a lot, because setup of Bluetooth is the unpredictable part of our work: if could not manage it then we would not be able to do anything.

Also we told about limitations which we discovered in LeJOS after deeper digging into it. For instance, there is no "swicth" operator at all, there can be only 64 fields in a class, there are problems with big numbers (arithmetic operations with long data type are not supported) and the 4th sensor port doesn't work with some types of sensors. The main limitation of LeJOS which should be always kept in mind is there is no garbage collector in it. So you should not create any object instances repeatedly (in loops or in listener methods), otherwise you will get an out of memory. But still LeJOS seems to be powerfull tool because of threads and synchronization support, which makes it usual for Java-programmers.

It is a last seminar before the final one so we have a lot of work to do during holidays...