1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Alpha Android Screen Grabber

Discussion in 'Development' started by abrenoch, 16 February 2018.

  1. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    ANDROID SCREEN GRABBER

    This application will use the built-in android APIs for capturing screen content and delivering that data to a hyperion instance. Unfortunately this also means DRM protected sources such as netflix are not currently supported. This issue may be able to be circumvented with root access but I can't confirm anything like that yet. This has only been tested on 2 devices so far (Pixel XL & 2017 Shield TV), so I would be interested in hearing how well this works on other devices!

    This is an alpha build - so I'm sure there will be some unexpected behavior. Please report any issues here:
    https://github.com/abrenoch/hyperion-android-grabber/issues

    There are some different settings for the capture rate, I suggest trying a few out to see what works best for you (I'll post a poll later).

    Please see the 'releases' page on github for an apk:
    https://github.com/abrenoch/hyperion-android-grabber/releases

    Here is a link to the Alpha test-channel on google play:
    https://play.google.com/apps/testing/com.abrenoch.hyperiongrabber
    Be sure to join the google plus group before using the link above (afraid this is the only way without needing people's emails)
    https://plus.google.com/communities/101526321905444484496


    Enjoy!
     
    Last edited: 24 August 2018
    • Like Like x 2
  2. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    ## [v0.5-beta]
    ### Changes
    - Added the ability to send only the average color of the screen
    - French translation
    - Norwegian translation
    - Czech translation
    - German translation
    - Dutch translation
    - Partial Russian translation
    - Partial Spanish translation
    - Removed openGL grabber option
    - Added toggle grabber activity shortcut
    - LEDs will now be cleared when rebooting or shutting down

    ### Fixed
    - Lights now clear (if running) when shutting down
    - Assertion bug in TV settings
    - Possible null intent when starting grabber
    - OOM bug

    ## [v0.4-alpha]
    ### Changes
    - Start grabber on device boot
    - Added some eye candy for when grabber is started
    - General UI tweaks (tv & mobile)
    - Reconnect behavior implemented for mobile build
    - New connection wizard
    - New settings/connection page (tv build)
    - Quick settings tile to toggle grabber (mobile build)
    - Screen orientation change updates grabber
    - Configurable grabber image quality
    - Pressing the notification will now return to the app's main activity

    ### Fixed
    - Grabber would fail to resume when waking device
    - OpenGL grabber sometimes halting immediately after starting screen grab
    - Default grabber failing to send data the first time it is turned on
    - Grabber not stoping when the host is unreachable
    - Aspect ratio of grabbed image being slightly off
    - OOM bug

    ## [v0.2-alpha]
    ### Changes
    - App Icon
    - Fancy toggle button
    - Bug fixes
    - New Grabber (old grabber can be enabled in the settings)

    ### Known Bugs
    - OpenGL grabber will sometimes hang when started, making the lights unresponsive. Quitting the app and starting again generally fixes the problem.
    - New grabber fails to send any data the first time it is initialized. Turning off and back on one more time seems to fix the problem.
     
    Last edited: 14 October 2018
  3. thommy86

    thommy86 Member

    Messages:
    55
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino, +PhilipsHue
    Tested it on a miBox.. I was not able to play YouTube, the grabber hang shortly after starting yt. Also the yt video became very small in the left top corner. I had todo a reboot to restart the grabber.

    I also tried to have a look at the code, but I was missing some files. Will have a look later.

    Still a great start, this will helpout a lot of people!
     
  4. sMode

    sMode New Member

    Messages:
    4
    Hardware:
    RPi3
    Hey,

    thanks for your work and for this App . I tested it on a Shield 2017, and it worked pretty good!
    Unfortunately the app crashed after about 30 minutes while watching a 4K Film with VLC Player.

    Also tested: Youtube Worked
     
  5. sMode

    sMode New Member

    Messages:
    4
    Hardware:
    RPi3
    Hello,
    thank you for this amazing app.

    I tested it on a Nvidia Shield 2017 and it worked pretty good.
    Unfortunately the App crashed after about 30 Minutes while watching a 4K HDR Movie with VLC Player.

    Also tested Youtube and it work, even with 4K
    Netflix/Amazon, dosnt work due to the fact of the SecureFlag ?!

    Hope this could help you a bit.

    Great Work !
     
  6. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    Glad to hear it is mostly functional! You are correct in that the secure flag would prevent it from working, but I'll try to root eventually and see if that helps at all. I'll have to dig up some HDR files to try out... If you have any (legal) links that can be shared to content to test please do! For the crashing, I'll have to try to reproduce on my end to say for sure. I'll try to add a way to share logs or something as debugging hardware I don't own isn't something I'm familiar with!

    What version of Android is your mibox running? There may be some discrepancies in the way different versions of android launch services that may make the grabber more prone to being killed... But the problem with YouTube becoming small definitely sounds strange - I'm not certain how the grabber would invoke behavior like that to be honest. If you can find a way to reproduce that it would be really helpful!
     
  7. sMode

    sMode New Member

    Messages:
    4
    Hardware:
    RPi3
  8. Flint

    Flint New Member

    Messages:
    4
    Hardware:
    RPi3
    hey there,

    thank you for working on this app!!
    I would like to test it on my Sony bravia TV.
    Unfortunately there are no LEDs connected to it yet. Is there a way to check if the app works properly on it without a complete set up?
     
  9. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    Sort of... But the only way to know how fluid it plays is to actually run the lights with the screen. The 'test' basically creates a series of image files that are essentially the same pixel data being sent to hyperion (really small compressed images). That works alright if you are trying to see if it will actually grab the screen content, but doesn't tell you much about how smoothly the lights run and with how much latency - for that I can't really imagine a replacement for actually trying it out!

    Just curious, for anyone having trouble, what is the capture rate you are running at?
     
  10. Flint

    Flint New Member

    Messages:
    4
    Hardware:
    RPi3
    thanks for your response!


    I see,
    Lets say, for now, all i want to know is if it grabs the screen content. What do i have to do..?
    I ve got the Raspberry running with libreelec, hyperion serivce on it and the screen grabber running on android TV.
    IPs and Ports are configured, just not sure how to look at those grabbed images..
     
  11. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    I'll have to re-enable the code that produces those images - it is not utilized at all in the current alpha release! I'm thinking I will have an option to save a frame to the device every 5 seconds for those who want to test it without a hyperion setup. Again it would say nothing for how responsive your setup is, but would allow you to find out if it will work with specific apps or not (for what its worth everything except Netflix has worked for me, but I don't run much of a variety).

    Unrelated note; I found a user on github who apparently had began work on an android grabber as well. His code for grabbing the data from a surface is a bit different than the code I'm using, so I'm curious to see if that technique would work better for some users here. Once I get a chance to implement and try that (and confirm it works obviously), I'll post another download for people to try!
     
  12. sl81

    sl81 New Member

    Messages:
    17
    Hardware:
    RPi2, 32/64bit, +Arduino
    Hello, i've been watching this developement for some time. I'm not much good with code so not much usefull in the ways of contributing to the project in that way, however i would be willing to build a system for testing/debug and such. If this is needed. How is develeopment going? I cant wait to toss all my splitters and converters in the trash really :)
     
  13. thommy86

    thommy86 Member

    Messages:
    55
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino, +PhilipsHue
    Finally had some time to make a video and upload it. As you will see, the grabber will crash after starting YouTube. The leds are flashing afterwards and YouTube shows strange behavior.

    The Mi box is running Android 6.

     
  14. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    For me personally, the app is actually working very well! I had went about a week without turning on my LEDs (I have them hooked to a smart outlet so I can turn them on/off with my google home), and to my surprise the grabber was still running just fine once the lights turned on. I've found a few bugs that would make the grabber service act a bit funky, like if you tried to restart the service after it already started running once it would act weird, but I think I sorted that out. A few more details to sort out an I'll have a second alpha build to post, hopefully in the next few days!


    Not able to see that video! But I did find an alternative method of reading the screen surface data that doesn't utilize openGL, I'm just struggling to shift the buffer into RGB data for hyperion right now... I'm honestly not sure yet if that will be an improvement or not, but I would certainly be interested in seeing if it helps with the mi box!
     
  15. Flint

    Flint New Member

    Messages:
    4
    Hardware:
    RPi3
    Just got my LEDs on that Sony Bravia Android TV. :)

    I installed the app, but couldnt find any icon on the homescreen. I finally found it in androids settings menu...

    Unfortunately it doesnt work. The leds are staying dark :/
    Is there anything more to configure besides hyperionserver, port, etc.? Maybe something special in android settings?

    Edit: just restartet the whole System and the grabber app... now all the LEDs are constantly white :D
     
    Last edited: 15 March 2018
  16. abrenoch

    abrenoch Android Grabber Developer

    Messages:
    89
    Hardware:
    RPi1/Zero, RPi2
    Hey Flint, glad to hear they at least are turning white haha. There could be a couple reasons for that, I've noticed on my setup occasionally when starting the grabber the lights will light-up white (which is the screen data from the hyperion app), but get stuck there and don't respond when I leave the app. Only workaround I've found is to try restarting the service a few times and it eventually starts working - which is a little mysterious to me. However the current posted alpha has issues stopping/starting the service from the app, so the little toggle switch doesn't actually stop the grabber. On a phone there is a notification item for the grabber with an exit button that does work, but there is no notification menu in android TV! I do have that fixed already so maybe I'll just post that fixed apk when I get a moment to.
     
  17. BETEP-ok

    BETEP-ok New Member

    Messages:
    5
    Hardware:
    RPi1/Zero, RPi2, RPi3

    what is the BRAVIA you are using?
    My panel is 55XD8577 and I also have issues- the grabber only working in menu and turns dark in Kodi, Moonlight, Youtube and so on.

    feels like Sony have some specific way showing movie video layer )
     
  18. TPmodding

    TPmodding Administrator Staff Member Administrator

    Messages:
    925
    Hardware:
    RPi1/Zero, RPi2, RPi3, +Arduino, +nodeMCU/ESP8266
    btw there is a kodi plugin for netflix...and it works with ambilight...
     
  19. Gershy13

    Gershy13 New Member

    Messages:
    28
    Hardware:
    RPi2
    Hey, I just found this thread... I have a shield TV 2017, will this app work with it and give me a Hyperion capture while still allowing me to get 4k 60 HDR output? And how would I go about setting it up?
     
  20. BETEP-ok

    BETEP-ok New Member

    Messages:
    5
    Hardware:
    RPi1/Zero, RPi2, RPi3
    just give it a try.
    https://github.com/abrenoch/hyperion-android-grabber/releases

    download-install APK - punch in simple settings - enjoy!

    in the best case- you have your less flickering.

    In my case - only software rendered elements change colours of the LEDs.
    Hardware rendered video layer not captured by the grabber as far as I get (((

    waiting patiently for the second alfa release - abrenoch is working on alternative grabber technique. ))))