My work is a demonstration of a mouse motion gesture recognition system using Dynamic HMM. It is developed in Java.

Code Available @ : https://github.com/gtiwari333/mouse-gesture-recognition-java-hidden-markov-model

**Main References were :**

**DEMO VIDEO**

**Details :**

The

**gesture features**is composed as

- Normalized Distance of current point relative to Centre of Gravity of all points
- Angle of current point with respect to CG
- Angle of current point with respect to Initial Pt
- Angle of current point with respect to End Pt

- Angle of current point with respect to Min-x and Min-y point
- Angle of current point with respect to Min-x and Max-y point
- Angle of current point with respect to Max-x and Min-y point
- Angle of current point with respect to Max-x and Max-y point

**Features of System :**

- Invariant with size of gesture drawing between training and testing
- High performance : Greater than 95%. (Detail Analysis is remaining)
- Tested with 30 different gestures (mostly composed of straight lines and few curved lines like S)
- Trained with 1040 Training data.
- Quantization of angles while composing features
- Orientation sensitive

**Implementation**

K-means, Vector Quantization, CodeBook Generation, Baum-Welch training, Viterbi Decoding : Algorithm codes were taken from my earlier project :

**Text-prompted Remote Speaker Authentication.**

Gesture Feature Capture Panel coded using MouseMotionListener and MouseListener, which enables capturing of drawing points with time data. The time information associated with each pixel is required during calculation of velocity feature and replaying the captured feature.

Verify/Recognize, Adding new Gesture and/or adding training data, Training the CodeBook and HMM can be done from main GUI.

**Choice of HMM constants**

- NUM_STATES (number of state in the model) = 4
- NUM_SYMBOLS (size of CodeBook) =64

**Result**

**I thought there was a nonzero transition probability to the success state. And I found it to be true.****Next step in this project :**I am planning to enhance it by integrating with OpenCV for Hand Gesture Recognition.

What a cool project.

ReplyDeleteI want to do something with this theme (mouse gesture recognition with HMM ) for my study.

What would you advise me ? How deep have I step into the actual Hidden Markov Model theory and its mathematics ?

Hi,

ReplyDeleteGood job, really interesting.

Do you have a PDF report that explains how you applied HMM to analyse the mouse movement ,

Thank you

I haven't prepared any report for this.

DeleteBut I have briefly summarized the Feature Extraction part in my blog above. You can easily understand the implementation my reading the code as well.

Once you extracted the features, the rest of logic to implement HMM is same. You can read any paper on HMM.

You can find the good articles on HMM here :

http://ganeshtiwaridotcomdotnp.blogspot.com/2011/08/list-of-good-references-for-speech.html

Hey there, as a beginner I'm really looking on what is the exact use of codebook. I tried to run the application without generating codebook and it worked. Could you please elaborate on the same?

ReplyDelete