CEP Capstone Thing???? by Randumb Person

[IN USE] Possible Idea #1: Virtual Keyboard (Object Detection)

randumbperson2003 — 2019-01-22 08:46:41 UTC

Hardware Setup: RPi 3 + Camera + Display on a stand looking over a piece of paper (+ compute stick??)

The camera detects fingertips and also determines when the finger is pressed and when it is not.

Then based on the location of where the fingers press down on the paper (calibrated on keyboard init), you can simulate a keyboard with no moving parts and can easily be replaced!

Better than laptop and mech keyboards (die from water), and even touchscreen keyboards (stop working when water is around).

Possible ways for improvement: vibrating motors for haptic feedback, compute stick to reduce latency

Possible Idea #2: Word Counter (Image Processing)

randumbperson2003 — 2019-01-29 07:49:36 UTC

Hardware Setup: literally nothing

Use ML to detect letters, spaces, punctuation to make a word counter + some more features (grammar checker?)

Possible Idea #3: Myoband Robot Arm (Signal Classification)

randumbperson2003 — 2019-01-29 07:49:55 UTC

Hardware Setup: Myoband and Robot Arm and Windows PC

Backup Idea: Use EMG signals from a Myoband in conjunction with an ML classifier to control an Arduino-powered Robot Arm

Full Proposal (KEYRIPPER)

randumbperson2003 — 2019-01-30 05:34:35 UTC

So...virtual keyboard. No project is complete without a punny name so I'll call it KeyRIpper.

So here are all the parts:
1. Fingertip detection
2. Finger tap detection
3. Paper detection / 3D plotting
4. Software (macros, linking taps on the paper to keyboard strokes, and keyboard reprogrammability)
5* . Haptics, latency reduction & optimization

Basic Software Outline

randumbperson2003 — 2019-02-19 15:12:33 UTC

Use to extract important features. From the features extracted, form a to detect the fingertips and where they are located.

A similar process will need to be used for the shadows casted by the fingers (and when the tips of the shadow = the tip of the finger, it's a tap). I will likely just add the shadows to the existing hand shape and run the existing SVM, but whether this will work is to be decided.

Final Submission

randumbperson2003 — 2019-07-31 16:19:22 UTC

Slides (explaining some pretty big changes to the original plan here): https://docs.google.com/presentation/d/1FPyrphvq9YCNzQNWjrRN3jxA59bF_wV1WJcHO9If5cw/edit?usp=sharing

Code (although it's messy): https://github.com/HTY2003/KiBoard