Hyperion NG: Architecture Discussion: Adding "great" camera support

Hi,

BACKGROUND

Target: Best quality Hyperion NG with working with ALL content (smart TV, streaming video etc...)

As known, a camera is "the" solution of that problem, but it is tricky.

The advanates and disadvantages has been discussed....

Since a USB camera /dev/video0 delivers this: (camera mounted at the ceiling ~1.5m away from TV)



I do not want to fix this by fancy LED configuration, what basically works.

A camera can only work in good conitions.

In my case: Dark room, no reflection, 4K HDR TV, decent USB camera (50€ class, not 5€).

That is a good starting point.

Additional "fancy" camera processing is required... a real "NG Feature" ;-)

This is not for cheap.

1. Good USB camera ( --> READY)
2. Horsepower for processing (Raspi 4) ( --> READY)
3. Dewarping of image (--> Need rectangle image as default Hyperion processing) (MISSING)
4. Full camera control (Exposure control, whitebalance, color calibration etc) (MISSING)

Good thing: All framre grabber will also benefit from that

WHAT CAN BE DONE

A Raspi 4 with a quad core ARM can do a lot.
Technical (driver, C++ code base) all is available in the open source community.

I want NO compromise on quality. In the past there has been performance bottlenecks,
but that is no longer the case .. with Raspi 4.

So, the idea is to add "native" USB camera support to Hyperion NG

Native means, a Hyperion specific "image processor" that does fix / control ANY problem of
any given /dev/video0 USB video camera or frame grabbers.

HOW CAN THIS BEE ADDED

I suggest to add a separate software module that does ALL the camera / video preprocessing.

The idea is, that it takes the /dev/videoxxx souce ( USB camera, Camera, Frame grabber).

It does all processing and delivers than the frame buffer to Hyperion as addtional video source.

WHAT CAMERA / VIDEO CONTROL IS REQUIRED

To Hyperion NG, there will be added a separate control for the camera / preprocessor

It offers the at least the following interfaces

Capture:

• Device selection
• Capture format (Resolution)
• Cropping

Exposure (only on camera devices, not on frame grabber)

• Framerate / Exposure
• Gain
• Possibly "smart autocontrol"

Color

• Whitebalance
• Contrast
• Hue
• Luninance
• Gamma
• (Lut)

Geometry

• Trapez correction (4 point)
• Dewarping (fish eye) (bezier curves)

Post processing

• Denoise
• Prefilter (smoothing)
• Output format (downsize)

USER INTERFACE

The user interface is a separate screen in the NG interface.

It shows all of the controlls above.

For calibration / settings it may be a good idea to build a small QT live gui.
This GUI runs native on the Raspi showing a live image on the HDMI output.

A "simple" version may be done with the web interface... I have no idea how this can be done.

The user will do 3 steps:

1. Enable "Preprocessor" (instead of /dev/video... or intern)
--> This tells Hyperion NG to take the frame buffer from the video processor
2. Go to Preprocessor
--> Make default seetings (Device, format....)
--> Calibrate geometry
--> Calibrate image quality
3. Optional: Run special software to calibrate the preprocessor


DISCUSSION

That is the "plan". ,, Please coment below.

Questions:

• It this approach feasable?
• Architecture / integration well thought?
• Missing features?
• What is overseen?

Thanks, Marko

 

U are making it more complicated than it really needs, from my understanding cams are already working with hyperion, at least those that show up in v4l2, but this can be configured.
The only thing we really need is trapezoid transform from capture source and some crop.
It really very simple and any rpi has the power to do it.

 

Hi Andrew,

my webcam works with given software, but its bad.

quote: The only thing we really need is trapezoid transform from capture source and some crop.

OK, and where and how can this be added to the existing system?

And: Important is good camera controll got get a very good image (Exposure? Whitebalance)....

that does not come with Hyperion yer, or am I missing some?

Marko

 

Well i dont know, but the trapezoid transform should be added to v4l2 device options i think as this will be used with cameras and they are v4l devices, this is the first and most needed thing, everything else is optional as good camera will work they way it should out of the box.
I have seen some trapezoid PR here https://github.com/hyperion-project/hyperion.ng/pull/656 but i dont know if this is it
A good camera shouldn't need any additional setting, your camera works but is bad, bad how?