Is this possible?!?!

I am planning on building a robot and have it do the following things:

• Is run with a Raspberry Pi 2 and using all Matlab/Simulink scripts
• Is streaming stereo-graphic pictures from 2 USB webcams
• Is taking the stereo-graphic images and locate objects and their distances
• Using that information to make and implement decisions based on its surrounding
• I would also like to program some functions of the robot that might not be possible in Matlab, and would need to be done in python.
• Do all of that without too much delay

I think I would need to use the Raspberry pi toolbox, as well as the image processing, image acquisition, and machine vision toolboxes. Anything else?

Is their a better place for this type of question?

Any little tip or hint that might point me in a better direction would be extremely appreciated! Thanks for your feedback!

Answer 1

[Martin Schätz]

Hi, if you ar not familiar yet with Computer Vision Toolbox i suggest to look at Stereo Calibration And Scene Reconstructio Example . That will answer a lot about your first three points.

With every computer it will depend a lot on the resolution of your web cams. That will influence the speed of getting the depth info. With resolution of something like 800x600 I gues it might be realistic to get depth data few times per second on Raspberry Pi 2. The images in the example are 900x1600 download trial if you dont have access to this toolbox and time it with tic toc (my guess is it will take around 1 second). And then you will have to look for objects and stuff in this depth map and make decisions based on that.

Also Raspberry Pi 2 has quad core processor and i am not sure if MatLab is capable of using full potential of multiple core processor. To speed this up C would be better.

But you actually can skip these 3 parts and use Kinect instead (or any other depth sensor, but Kinect should be cheapest). I suggest Kinect v1 (from Xbox 360) because of the method used for its depth sensor (if you dont want to use it outside). It will get you the depth data in relatively good resolution at 30 FPS without using much of the Raspberrys processor. And you will also find libraries for Python for it. This might be inspiring for you.

[Ben Messing]

Thanks Martin for the feedback.

I will definitely look more into the Computer Vision Toolbox. There is a lot of great support to be found all over the Internet.

As for the decision making, two or even maybe one time a second would probably be sufficient. I am pretty sure that when the Ras Pi takes a picture, the resolution can be specified. It should not be that hard to find the optimal settings.

Some questions about the coding aspect: In your 3rd paragraph you mention C being better. Would you recommend not using MatLab at all? Or is their an easy way to combine MatLab with C? would C be better than Python because C is compiler based and would be easier to mulithread with? (I say "easier" with hesitation because I am neither experienced in C or multithreading. I rather code in Java or Python, but these are both interpreted languages, and inherently slower at such things)

Really newbie question: Would the final code would mostly written in C/Python, with some parts somehow interacting with a MatLab script? Or is this not the correct setup?

Using the Kinect sensor sounds REALLY COOL, but my robot happens to be more of an outside thing so oh well.

Thanks again for the truly excellent feedback. I really appreciate it.

[Martin Schätz]

Hi, you are welcome. I sugested C becouse it is mcuh faster because, as you wrote, it is compiler based. But it is always better to stay with what you know. And I also forgot to search for this . If MatLab has support for Ras Pi 2 then stay with MatLab. It is much easier to learn. As alternative, you can use MatLab functions from Java and vice versa. ( First search hit. I am a bit familiar with use Java in MatLab) But i can't say much about it from my experience.

As for Python/MatLab i found some documentation. But don't have any experiences with it.

From what I did learn about support of Rsp Pi 2 from Matlab, you can prototype everything there, and then you can do the final program in Simulink and deploy it directly to Raspbery. Here is webinar for it.

MatLab will speed up develepment for you. If you know basics you can do it relatively fast. In Python or Java it depends on how it will work in the end, when you deploy your solution.