Check out the air gesture and real-time rendering engine… ready for integration in your own app. Give it a try and tell us what you think of it!

Get your copy while it is hot!

• a selection of 3D interior designs
• 3D zoom for close-up inspection
• real-time 3D rendering
• smooth pan in full 360 degrees
• walk in the scene – literally!

Signal processing and FFT

Fast-Fourier transforms (FFT) are widely used in digital signal processing. RealityFrontier has been facing with two needs: lowering the energy consumption for mobile applications as well as increasing the processing throughput for real-time applications.

The computation is performed by a small-form factor PC. It is housing a NVIDIA card programmed using the CUDA toolkit. The throughput is as much as 50x compared to the same FFT computation performed on a similarly priced CPU board. It is still 10x higher than a computation optimized for an Intel multi-core CPU. (See our analysis below)

An example of use of FFT is in mobile robotics. The 2D FFT of an image provides for instance a high-pass filters in an image processing pipeline.

CoroWare, a partner with RealityFrontier, has been speeding up the signal processing of its mobile robots by using the RealityFrontier acceleration appliance.

Appliance specifications

RealityFrontier appliances are aimed at speeding up the computation of FFT while keeping the appliance’s electric consumption to a minimum. RealityFrontier offers two FFT acceleration appliances.

	Embedded systems	GPU Stream processors Power consumption Cooling CUDA Toolkit Speed-up (2)	ION 2 16 35 Watts Fan-less 3.2 up to 10
	For the lab	GPU Stream processors Power consumption Cooling CUDA Toolkit Speed-up (2)	GTS 450 (1) 192 350 Watts 2 fans 3.2 up to 50

NVIDIA CUDA Toolkit 3.2

NVIDIA has recently updated its programming CUDA Toolkit. The CUDA Toolkit 3.2 provides many performance improvements over its previous iterations, making it a platform of choice for RealityFrontier when compared to OpenCL or specialized math libraries such as the Intel MKL library.

The latest CUDA Toolkit brings some noticeable improvement for signal processing. The CUDA FFT library (known as CUFFT) is delivering a 300% increase in performance compared to the previous Toolkit version. The benefits gained with the CUDA Toolkit 3.2 apply only to the lab version of RealityFrontier compute appliance – as only that version is using the power-hungry Fermi architecture.

Contrasting NVIDIA to Intel

NVIDIA has published how its latest solution compares with the one from Intel, its Nemesis in the many-core GPU war against multi-core CPU. The CUDA Toolkit brings 2 to 10 times faster performances when compared to Intel’s MKL library. NVIDIA compares as well favorably in terms of software licence cost. Intel has priced its MKL at USD 399.00. NVIDIA does not charge for its CUDA Toolkit 3.2.

One will have to use the latest architectures from both vendors to obtain these figures: the Fermi architecture in the case of NVIDIA and the Xeon™ or Core™ processor family in the case of Intel (or a processor compatible with Intel’s Streaming SIMD Extensions instruction set).

Let’s evaluate how RealityFrontier’s compute appliance will perform if one would decide to deploy the Intel’s MKL on it. Only the lab version of our compute appliance can support the MKL. The embedded system version use a ION 2 that is not SSE capable. 

The compute appliance is powered by a dual-core CPU. Each of the CPU cores has two physical threads, bringing the total to 4 physical threads. Intel has published a whitepaper that shows an almost linear increase of performance with the number of threads up to 8. With 4 threads, we would then expect a 4-time speed up by using the MKL.

The GPU brings a far higher speed-up than the CPU even when using the Intel MKL.

Other Toolkit 3.2 improvements

Does your application need other applied math improvement? Using the latest CUDA Toolkit will bring some benefits as well. NVIDIA has announced that the new CUDA version sports a similar 300% improvement in performance compared to the previous version for the CUDA BLAS library, making it 8 times faster than MKL (useful for linear algebra such as matrix multiplication). Other improvements are in CURAND making this library 10 to 20 times faster than MKL (useful for random number generation). Finally, the latest version of CUSPARSE delivers blazing fast routines that operate on sparse matrices and that are between 6 and 30 times faster than MKL.

Additionally, the CUDA Toolkit 3.2 now comes with built-in support for H.264 encoding and decoding. Both the support for H.264 and the BLAS performance improvement are of particular importance in the field of video processing. RealityFrontier’s customer will benefit from these improvement when using video stream computation on mobile devices or in the lab.

Benefits

Using GPU cards to accelerate FFT is believed to provide one of the best throughput per watt as well as the most cost effective solution. Such a massively parallel architecture is a particularly good fit for signal processing.

RealityFrontier is primarily offering these compute appliances as part of its CUDA consulting service. The appliances can also be ordered as stand-alone equipments by those customers who wish to code their own CUDA algorithms with a pre-configured system.

This is an early availability announcement. Contact us for pricing information.

More information

CUDA Toolkit 3.2. NVIDIA programming toolkit (November 2010).
Intel MKT. Intel math library optimized for multi-core CPU.

Here’s a selection of recent events that RealityFrontier attended.

• mHealth Summit 2010 – where the future of mobile health is taking shape
• Tele-health and ergonomics – an inspiration for better product design
• Healthcare IT in Boston – local networking event
• 2010 Connected Health – improving the quality of healthcare with robotics
• MassTLC Innovation 2010 – future of telemedicine

It was great to meet with other companies that are using video for tele-health.

Video in tele-health, really?

Image processing and video processing have been used extensively in the medical environment. Our goal at RealityFrontier is to get medical imaging out of the lab. Video content is collected in hospitals, communicated over long distance in telemedicine scenarios, used for tele-health in elderly care, and more.
We are not alone in this search for new design ideas. Recent work has been conducted to use consumer web cam in order to measure health indicator such as pulse and respiration. For instance the CardioCam project is able to measure the pulse of several people simultaneously using slight variation in skin tone captured with a simple web cam. Watch their video.

Similarly, machine vision is used routinely by ergonomics labs to study health factors with the help of algorithms for posture tracking, gaze tracking, gesture recognition, etc. Other type of sensors can be used that eventually produce a video stream, such as ultrasound sensors.

mHealth Summit 2010

The mobile health summit attracted 2,000 visitors and more than 100 exhibitors for a two-day event in Washington, DC, in November 2010. The summit was the occasion to listen to some great talks and discussion panels, such as the “Lessons Learned across the Globe”. This panel included the United Nations Foundation, university researchers, communication service providers such as Orange, and health service organizations. Watch the video.

The exhibit floor was packed with mobile device manufacturers and research projects. The summit was clearly targeted at a professional crowd although most of the devices manufacturers were addressing the needs of individuals and their care givers. Many of the advances in mobile health are funded by government grants. Industrial alliances are represented primarily in the Telecom field and as well through standardization bodies.

Some of the exhibiting companies were demonstrating the use of video-capable devices for telepresence, remote diagnostics and medical data processing. Their products and research projects are of particular interest from a video content analysis standpoint.

NETRA – when the eye itself is acting as a measuring device.

MIT’s camera culture group describes further how they intend to distribute a Smartphone for affordable optometry.

MobiSante – when mobile health is really mobile.

MobiSante, a Redmond-based firm has developed over the past three years the first ever Smartphone based ultrasound imaging system. The device can provide a live image of various organs such as kidneys, etc. It can be used in a remote location without much training. The images can be streamed to a specialist using the Smartphone wireless connection. Watch the video.

LookTel – a visual electronic assistant for visually impaired or blind people

The LookTel application by IPPLEX transforms a Smartphone for very compelling scenarios: object recognition, OCR, object and landmark tagging, etc. What their video.

Data acquisition and processing can face some challenges. For instance, how to optimize the power consumption of a mobile device running for long period of time? That’s what Wockets from MIT presented with a poster on “enabling power-efficient continuous sensing on mobile phones”.

The mHealth Summit 2010 helped us at RealityFrontier in positioning our on-line service for video processing at the crossroad between data acquisition on the mobile device and data processing in the datacenter. Furthermore, video content processing should go side by side with multi-sensor data fusion and low-power scenarios.

Tele-health and ergonomics

RealityFrontier was at the NEC HFES 2010 event organized by the New England Chapter of the Human Factors and Ergonomics Society.

The keynote speaker, John Moore, MD, is working on the participation of the patient in their care at the MIT’s New Media Medicine lab. Opening of the ergonomics event by John Moore: video #1, and #2. 

Many of the presentations were underlining the use of video technology in group studies. This event helped RealityFrontier in discovering another area where video content analysis can be put to fruition.

Healthcare IT in Boston

RealityFrontier attended this networking event profiling Healthcare IT companies. Several companies local to the Boston area delivered a 2-minute elevator pitch to the audience. Watch the presentations:

Athenahealth

Arcadia Solutions

Sproxil

Relay Technology Management

HanGenix

Versik Health

Healthcare Revenue Strategies

Fivi

Neumitra

Gene Network Sciences

Medidata

The event was organized by Propel Careers and the Massachusetts Medical Devices Journal in partnership with the Boston Regional Entrepreneurship Week. The event is one among many hosted by the Microsoft New England R&D center in Cambridge, MA.

Some of the emerging trends in the Healthcare IT industry are the increasing role of analytics and Clinical Decision Support Systems (CDSS). As an example, RealityFrontier work with video content assets could play a role in CDSS as they gain in adoption.

Other events

RealityFrontier attended two other events related to tele-health recently: a half day session on the topic of “Improving the Quality of HealthCare with Robotics” organized by the Center for Connected Health and Future of telemedecine, a discussion held at the MassTLC Innovation 2010.

RealityFrontier & tele-health

Tele-health and mobile-health are both using video content for communication and diagnostics purposes. Mobile devices and Smartphones are recording and streaming videos from the patient’s hands. Those videos are consumed by care providers, processed through service providers or yet used by the individuals themselves. The videos are no different than any other videos in that they contain information that can be automatically indexed, searched for and processed. Video content analysis is core to RealityFrontier’s service offering to medical device manufacturers.

More information

• Future of health, a report by PSFK , to position tele-health within a broader context
• Telepresence for tele-health, by CoroWare, one of RealityFrontier founding partner
• Video content analysis, for more information on RealityFrontier’s current work

An article by RealityFrontier
community@realityfrontier.com

Future of telemedecine

What helps in building up a telemedecine solution?

• A sales team needs to employ the services of a medical doctor as doctors are trusting doctors.
• The solution provider needs to get involved as closely as possible to medical trials.

 What are the key contributing factors for the adoption of a telemedecine solution?

• Facilitate the relationship between doctor and patients.
• Move data collected remotely where it belongs.

 What are the barriers to adoption? 

• Cost of entry for following up with some categories of patients such as diabetics, obese, etc.
• Cost of putting a solution together as it involves the participation of medical practitioners and technology providers. Seen from the medical team standpoint, it often costs less to work with college students.
• Reticence of medical professionals to adopting new technology as it is perceived as getting in the way of caring for patients.

 How to get into the telepresence market?

• Understanding the re-imbursement mechanism is a key factor as it drives the incentive for medical practices.
• A solution provider would have to build a model of a medical practice cash flow that would account for the new technology cost.
• Identifying the right buyer is critical and may require some consolidation. An individual doctor may not buy in.
• Understanding the end-to-end care scenario is also critical. The reason why a black box is useful for home healthcare is that nurses are working on a prospective pay where additional visits are not paid for.

 What are the key metrics?

• Improving the quality of care
• Reducing the cost of care

 What are examples of technology that best fits telemedecine?

• SMS is preferred as there are not that many Smartphones in use.
• It does not need to be real-time for some fields, such as dermatology.
• Any technology that helps in submitting measures for bonuses in the field of evidence-based medicine.
• Help in collecting enough data to define a norm for a particular scenario, and in identifying  when there is a variation from the norm.
• Note: HIE is orchestrating the anonymous collection of data.

 What is evidence-based medicine measured on?

• Primarily focused on processes rather than medical data
• Currently more used to compare performances of medical practices
• There is no mechanism currently that would be close to a consumer report

More Information

More about the unConference format.

Cameo appearance!  Minute 5’11” on Bill Warner’s video.

Video content analysis looks into a video footage, frame by frame, and identifies relevant information. It is different from video monitoring analytics or video viewing analytics. Video analytics is primarily used by security firms for real-time area surveillance. Video viewing analytics is used by media agencies for ads placement based on the video metadata. By contrast, video content analysis goes one step further: it looks into the video content itself, extracts key information and rebuild the storyline.

From a broader standpoint, the software industry is moving to a device + datacenter model. For the purpose of analysing video content, the mobile device offers the capability to acquire and display video while a series of on-line services are providing the video services. Let’s have a look at what the range of services readily available.

Video content analysis is a fragmented market: service providers are focused on particular scenarios they understand. The companies often have invested years in developing their technology. Some existing platforms are for instance Blinkx, Gotuit, Google or yet Viewdle.

• Launched in 2005, Blinkx went through two IPO and has a market capitalization of $350M. It indexed 35 million hours of videos through 720+ media partners.
• Google rescently launched their Goggle service that let users search the web with pictures they take from an Android or an iPhone.
• GotuitMedia is a social video tagging SaaS application. The platform tags video content and lets the user access the video segments that matter to them.
• Viewdle is a digital media platform that offers face recognition. The service analyzes each frame in order to create a index that will be used to monetize video assets.

How does RealityFrontier compare to these services? Firstly, we are in the business of assisting our customer with customized product realization – as opposed to building a generic platform. To that extend, we are reusing the services of larger players for custom scenarios. Our services are targeted to real-time analysis. We focus on enhancing the capability of mobile devices with on-line web services. Our video analysis leverage hardware acceleration, a technique that allows realtime processing. We go one step further by providing complex event processing on top of video content indexing.

Here’s an example to give an idea of the type of results a video search can provide. Let’s look up “augmented reality” on a Blinkx Wall.

The video-conference market is exploding with offerings. High-definition, low-cost, mobile are some of the current trends to name a few. RealityFrontier offers a tele-health solution through its founding partner CoroWare.

This first article in a series of three is describing our capabilities for tele-health. A second article will show the challenges that all vendors are still facing with the technology. A third and last article will outline the technology landscape and opportunities for product designers.

Video-conferencing on the desktop is affordable. It allows several specialists and care-givers to get together at the same time. The quality of the HD video is good enough to visualize medical images. Furthermore the video communication is secure. Getting all of these features bundled together would be truly unique on the market. The solution exists and is called VidyoHealth.

CoroWare ^(*) has been delivering a video-conferencing solution based on VidyoTechnology for two years. Joshua Sherman, Telepresence manager with CoroWare describes his work for tele-medecine:

“ I have been deploying VidyoTechnology for several geographically distributed medical practices. Being able to use wireless devices made a real difference to my customers. But what really moved things forward was the confidentiality of the communication. The solution is secured. ”

 _{(*) CoroWare is a founding partner of RealityFrontier.} 

CoroWare’s telepresence solution is a SaaS product available through a subscription. The advantage is that patients do not need to acquire specialized hardware. They are using their home computer and the VidyoHealth one-button solution. Medical practicioners are able to re-use their existing video-conferencing equipment.  

RealityFrontier is exploring new ways to deliver tele-health solutions and is referring to VidyoHealth as a standard of quality for video-conferencing needs in tele-health. One of the areas that RealityFrontier is working on is the automation of video content analysis.

More information

• Interested parties may contact Joshua by e-mail for a demonstration.
• Please contact Lloyd for custom tele-health implementations.
• CoroWare is a hosted service provider of Vidyo Technology. Check the VidyoHealth datasheets for tele-medecine and tele-health.
• Check the CoroCall subscription that bundles all in one low-cost solution.

Next articles in the series

• Challenges faced by telepresence vendors in the tele-health market (upcoming).
• Technology landscape and opportunities for product designers (upcoming).

Robotic platforms are constrained by a feedback loop that should execute in typically less than 1/10 second on a low electric power budget. 

One-stop system integration  

Connecting the robot to a remote compute appliance helps in off-boarding compute-intense operations. More electrical power is available to operate the robot payload such as outdoor drive base, laser range finder, 4-DOF arm, etc.

Compute-intense computations are executed remotely. The API dynamically routes a stream either to the local machine or to a remote machines. The API also encapsulates compute nodes within XML web services.

The API is ready for integration with OpenCV and Player. An upcoming version will support calls from OpenNVIDIA.

Benefits

RealityFrontier worked with CoroWare to enhance their Explorer and CoroBot products with CUDA. The compute appliance option allows designers and researchers to work on compute intense algorithms such as visual SLAM.

More information

• Check the conference coverage by the press.
• Have a peek at the 60 exhibtors in the Emerging Company Summit.  
• Watch Lloyd presenting our CUDA demo in the exhibition hall.

Datasheet

• Reality Frontier’s datasheet PDF

Video

November update: the article is published! 

The Linux Journal introduces the article in their latest issue:

” If all that seems a bit too fluffy for your technology taste, you might just love Alejandro Segovia’s in-depth piece on parallel computing with NVIDIA’s CUDA technology. Video cards are great for gaming, but it’s amazing how powerful they can be when you use them for straight up mathematical processing. Alejandro shows how to take advantage of the little powerhouse sitting inside your computer case. “

CoroWare, a founding member of RealityFrontier, sponsored the article in May 2010. Since then, the technology evolved. You will find an update and a behind the scene story on the CoroWare web site. A big thank you to CoroWare for the opportunity to write the article and to the Linux Journal for publishing it! 

Saturday, July 24, will be historic with the “Life in a Day” event. This event is a global challenge to record a day in your life, an event sponsored by big industry names reaching out to anyone with a video camera, worldwide. It will become the first user-generated feature film of its kind. http://www.youtube.com/lifeinaday

As part of a company specialized in HD video communication, we have the means to “document a single day on earth”. To support that event, I’d like to help in providing a HD Flip video-camera to anyone who will ask for one.

This offer is obviously of a marketing nature that starts with the “Life in a Day” event. It will continue throughout the entire year… meaning that you will be keeping your HD Flip video-camera throughout the year! For free.

Or almost free. What do I ask in exchange? Of course have fun with uploading your “life in a day” video between July 24 and July 31 on the event’s site. Then recording a 2 minutes video shot, every month on a technical topic of your choice.

Think about it… you will be participating in an historic cinematic experiment, you will have your personal Flip, and you will help us with one video shoot a month.

All you need to do at this point is to reply today!

Raphael

  
Life in a day… and beyond!
One year of video with a HD Flip camcorder.

Conditions to participate

• Respond as early as possible today or early this coming week.
• Limited to 10 HD Flip camcorders in total
• Friends and family can apply
• You agree to submit to CoroWare one video shoot per month on a topic of your choice
• You can opt-out at any time by just returning the camera
• At the end of the year, in July 2011, you keep the camera
• … and you will be part of historic event in any case with the “Life in a Day” event!