Optimizing screen share speed and robustness (SOLVED)

[Ahiru]

UPDATE 2023-03-28:  See Reply #13 post near end; we think this issue was due to using an older 5th gen iPad as the following device; an upgrade to a 9th gen iPad elimanated the screen share losses.

UPDATE #2 2023-07-13:  See Reply #19... we also found connecting the iPads to a WiFi router (no internet connection necessary) resolved repeated issues we had with establishing the shared connection.

Looking for info and advice on screen sharing with iOS (iPads)... in recent rehearsals broadcast screen sharing (app 4.2.14 on iOS 14.6) was so slow or failing to refresh it was almost unusable, whereas in the past it has been working fine (tolerable lags in updates from the main to secondary iPad).
(Our iPads are 5th & 6th generation aka 7,5 positioned only about 6 feet from each other)

Coincidentally I saw these release notes for your latest release 4.2.15 has some improvements for screen sharing, and we look forward to trying this update (probably at a gig later this week):
- Improved the transmission of screen sharing images from slow lead devices, and updated following devices to show whatever they receive of an incomplete image to indicate that the connection is still active.
- Updated the screen sharing function to resend the first screen image after an iOS device connects since it often skips the first image.

So, looking for general information to understand improve screen sharing:
- Can you elaborate on the 4.2.15 updates... were they necessary due to some change in iOS?
- In general has something changed over the last month or so in iOS or BH to make this less reliable?
- For broadcast screen sharing, any pointers to understanding the underlying mechanism, e.g. to understand is it better to disable BT if not in use, is it UDP-like or does it retransmit on failure, etc.?
- Any best practices for improving speed/reliability?
- Would it be faster / more robust if we switched from 'Display Screen' method to 'Follow Actions' method?  (Does that also use the same underlying transport as the screen broadcast?  Could it be worse, if some key event like 'start song' is dropped and thus no recovery opportunity until the next song?)

Thanks!

[arlo]

I got a small wave of bug reports about unreliable screen sharing in the last month. Because of the pandemic stopping a lot of band activity, it's hard to know what to attribute that to, but I'm guessing it's related to the app rewrites released in March (BandHelper) and May (Set List Maker) and is showing up now because more bands are starting to perform again. The networking approach hasn't changed, but it seems to run more slowly in the newer programming language, or be less tolerant of slow devices.

Of the bug reports I received, all involved using a lead device that was at least 4 and up to 8 years old, and all but one were using the latest iOS version, which means the devices were already somewhat burdened with running an iOS version much newer than the hardware. In my testing, I saw a clear improvement when using a 9 year old device, then a 7 year old device, then a 4 year old device. Some users have confirmed that switching devices around so the newest device was the lead device made a big improvement.

Both forms of Live Sharing (screen sharing and remote control) use the same networking approach on iOS and Android (multicast DNS for discovery, and socket connections for data transfer), but this is somewhat abstracted by the operating system and I don't directly control how devices connect to each other. iOS especially makes a point of abstracting whether wi-fi or Bluetooth is used. In an initial test, I found that turning off wi-fi to force Bluetooth was more reliable for screen sharing, but I didn't try again after my optimizations because those seemed to address the problem. In general I think the best advice is to leave both available and let iOS use whatever it thinks is best for the situation.

Remote control is generally faster and more reliable because it requires sending a tiny fraction of the data that screen sharing requires. Neither function verifies what data has been received or retransmits data that was not received. I could put that on my wish list, but it wouldn't be very useful other than for song selections since all the other data changes pretty quickly.

BTW, this post isn't asking about device discovery, but for future reference, the most common issue with that is a router configured to block intranet traffic. Also, iOS 14 introduced a privacy setting for Local Networking that the app offers to turn on when it first runs. If you decline this or turn it off later, device discovery won't work, and the app can't detect whether it has been turned off so it won't show an error message.

[Ahiru]

Thanks Arlo, this is a big help!  Indeed we've only rehearsed with sharing screens a couple of times in the last month, and our first gig since last year is this Friday ;-)  So this helps us understand it is not something else in our overall wireless environment.

Remote control is generally faster and more reliable because it requires sending a tiny fraction of the data that screen sharing requires. Neither function verifies what data has been received or retransmits data that was not received. I could put that on my wish list, but it wouldn't be very useful other than for song selections since all the other data changes pretty quickly.

I can imagine this could be tricky to add, but for our use case (everything automated) seems like just having guaranteed delivery for first and second song select should be enough to allow the following device to play a song independently and roughly in sync with the lead device.

BTW, this post isn't asking about device discovery, but for future reference, the most common issue with that is a router configured to block intranet traffic. Also, iOS 14 introduced a privacy setting for Local Networking that the app offers to turn on when it first runs. If you decline this or turn it off later, device discovery won't work, and the app can't detect whether it has been turned off so it won't show an error message.

Thanks... we haven't had any discovery issues; just lost broadcast screens.

[arlo]

I can put that on my wish list, but if you end up using that workflow, please let me know if it is ever actually a problem (i.e., if the song selection message ever gets lost).

[Bill Malchow]

user full info here.  thanks arlo for being so descriptive.

[Ahiru]

Resurrecting this topic....  We continue to have reliablity issues with screen broadcast or action broadcast using iPads that seem to follow this rough pattern (over the last year or so):
- Initial connection appears to work OK.
- However, the following device fails to display a screen fairly soon afterwards (perhaps within a minute).
- We connect again successfully.
- The following device might work fine for about 5 or 10 minutes, then fail again.
- We connect again successfully.
- The connection and screen sharing works without issues for the rest of the gig / session (hours)

The 'flaky at first, but finally settles down' behavior has repeated so often we wonder if there's some kind of a real pattern here... as if something in the connection takes multiple retries to get lucky and pick the most robust channels and/or config.

This is always with the latest BH and iOS, with an iPad 6th Generation (iPad7,5) as the lead device and iPad (iPad6,11) as the following device.

We see the same connection behaviors whether using screen broadcast or action broadcast, and whether the devices are connected to an active WiFi network or not.

Maybe our iPads (hardware) are just too old?  Mostly just wondering if this is a unique issue for us, or something others have seen and perhaps successfully addressed.

[arlo]

To clarify, are you talking about sharing actions or only sharing screens? You said both at one point, but most of this thread is about sharing screens.

Also, when the problem happens does the app show that a device has been disconnected? In older app versions you will see the number next to the live sharing button in the top toolbar change, and in recent app versions you will see an alert appear briefly when a device disconnects.

In my own testing, I've never seen a device randomly disconnect and I don't think I've seen transmitted actions get lost. But I sometimes see devices bog down when sharing screens if you're doing a lot of navigation on the lead device so that the screen changes frequently, or auto-scrolling, which I do not recommend with screen sharing. In those cases, the following devices can struggle to keep up with all the transmitted updates.

[Ahiru]

To clarify, are you talking about sharing actions or only sharing screens? You said both at one point, but most of this thread is about sharing screens.

Both... we mostly use sharing screens, but we've tried sharing actions a few times to see it was more robust, but saw no improvement.  It seems to not be related to the volume of transmission, as much as maintaining the connection.

Also, when the problem happens does the app show that a device has been disconnected? In older app versions you will see the number next to the live sharing button in the top toolbar change, and in recent app versions you will see an alert appear briefly when a device disconnects.

Usually the lead device eventually indicates it is disconnected, but perhaps about 30 to 60 seconds after my partner can tell he's not getting updates on his following device.  On my lead device I'll see the 'disconnected' popup and then the sharing icon in the upper right will go 'grayed'.  FYI I've attached a screen shot of the connection status during one session.  I can't correlate this to exactly what we were doing at the time so it's probably not that useful for diagnosis, but it seems like there are a lot of nearly instantaneous connect / disconnect / connect events that seem odd.

In my own testing, I've never seen a device randomly disconnect and I don't think I've seen transmitted actions get lost. But I sometimes see devices bog down when sharing screens if you're doing a lot of navigation on the lead device so that the screen changes frequently, or auto-scrolling, which I do not recommend with screen sharing. In those cases, the following devices can struggle to keep up with all the transmitted updates.

In our cases we're not doing a lot of navigation when the connection gets dropped... often just playing a song with an occasional scroll.

I suspect this is about something causing a lost connection (even while quiescent) more than a transmitted data volume issue.

But that's useful information if ours really an unusual situation... I'm believing it is time to try some different iPads!

[arlo]

Are you clicking the Refresh Broadcasting button in the Live Sharing status window? That would explain the sequence of events shown in your activity log: closed socket, removed peer, stopped screen broadcast, started screen broadcast. You should only use that button if following devices are trying to connect to your lead device and can't find it.

Also, it looks like you have Broadcast Screen and Broadcast Actions both turned on on the lead device. That shouldn't matter, but I would turn off whichever you're not using to simplify the troubleshooting.

[davelson]

I had similar issues with Screen Sharing.... Ended up disabling it. Most probably the lock ups were because we use Auto scroll on nearly every song. We now only use Follow Screen Actions - much better sync, rarely disconnects, and song selection is quickly duplicated on following device... Would be nice to have the following iPad also receive the auto scroll start command, but hey it ain't too hard to get the singer to push a soft button at start of song.

[Ahiru]

Are you clicking the Refresh Broadcasting button in the Live Sharing status window? That would explain the sequence of events shown in your activity log: closed socket, removed peer, stopped screen broadcast, started screen broadcast. You should only use that button if following devices are trying to connect to your lead device and can't find it.

I don't recall for sure, but that makes sense (and I know I've tried that Refresh button at some point in experiments).

Also, it looks like you have Broadcast Screen and Broadcast Actions both turned on on the lead device. That shouldn't matter, but I would turn off whichever you're not using to simplify the troubleshooting.

Yes, for a few passes I had both on (so perhaps also during this screen grab), but later I've always had only one on.

Anyway, not a particularly useful screen grab 

[Ahiru]

I had similar issues with Screen Sharing.... Ended up disabling it. Most probably the lock ups were because we use Auto scroll on nearly every song. We now only use Follow Screen Actions - much better sync, rarely disconnects, and song selection is quickly duplicated on following device... Would be nice to have the following iPad also receive the auto scroll start command, but hey it ain't too hard to get the singer to push a soft button at start of song.

Thanks for the feedback... yes I can see how Auto-Scroll would be tough for screen refreshes.  We use automated paging to custom markers for advancing lyrics, so a screen refresh occurs only every 15 to 30 seconds or so.

[arlo]

Would be nice to have the following iPad also receive the auto scroll start command, but hey it ain't too hard to get the singer to push a soft button at start of song.

If you start auto-scrolling on the lead device with a Song Selection layout action, or a Two- or Three-Fingered Tap layout action, or a remote control action (foot switch), you can set the following devices to start it from the same action and then the following devices will start auto-scrolling when the lead device does.

[Ahiru]

We've now replaced our 'following' iPad, which was an older 5th generation iPad with a newer 9th generation iPad and now it seems to be maintaining screen share connection flawlessly (whether using a 6th gen iPad or 9th gen iPad as the 'lead') in tests.

I don't know if this is an issue with the 5th generation iPad in general, or some failure in that specific iPad, but for now I'm marking this issue as SOLVED.

[arlo]

Good to know, thanks!