Guest Blog Post by Andrew Hoefling
The other week (10/2/2019) Microsoft had a special Surface event in New York City where they unveiled multiple dual-screen foldable devices. The Surface Neo, running a new Operating System, announced Windows 10X at the event, and the Surface Duo, a smaller device running Android OS. Both devices are being announced a full year prior to their release to give developers time to update their apps to support the dual-screen form factor of the new Surface devices.
Since then developers have been looking into APIs and asking the following questions:
- When will there be a public simulator or emulator?
- When can we get a development kit?
- Where is the documentation on the new APIs?
Panos Panay, Chief Product Officer at Microsoft, stated in an interview:
We’ve got probably a couple months before developers are going to have their hands on it and be running.
While we wait for the official documentation, I took it upon myself to start researching the Android APIs and how I would attempt to write a native Android App to support a dual screen device. As the research unfolded, I would share it on Twitter to discuss with others in the software development community.
Who am I?
My name is Andrew Hoefling and I’m a Lead Software Engineer at FileOnQ where I build mobile apps for police departments and first responders using .NET technologies such as Xamarin. I spend time sharing my knowledge and experience with anyone that wants to listen through my blog, speaking at conferences and discussing technology on Twitter.
What kind of authority do I have to talk on this topic?
I’m a developer that has been doing this for a while and I’m here to compile a list of documentation articles to explain some of my opinions and observations.
Starting to read up on the Android docs for dual window support and how this could apply to the #SurfaceDuo. According to the docs there will only ever be 1 activity in the RESUMED state, the other will act as STARTED until the user interacts.
Why is this important?
— Andrew Hoefling (@andrew_hoefling) October 4, 2019
Surface Duo & Galaxy Fold
The Surface Duo isn’t the only foldable device that developers need to support. The Galaxy Fold is another anticipated device that Samsung is building. Understanding the differences in these devices and how apps will interact is important as a mobile app developer.
Surface Duo
The Surface Duo is a foldable device that has 2 separate screens allowing it to fold 360 degrees. During the Surface event, the device showcased productivity use-cases where you could run different apps on the 2 screens at the same time, or the same app in a dual-screen mode.
Galaxy Fold
The Galaxy Fold takes a different approach from the Surface Duo where it can be folded to 180 degrees. When the device is opened you have a tablet-like screen with no seam down the middle since it is truly 1 screen.
Multi-Window App Support
Android has supported Multi-Window apps since Android 7.0. Yes you read that right, since Android 7.0.
What exactly is a Multi-Window App?
- Split Screen
But this isn’t really dual screen support like you would see on the Surface Duo?
Well technically that is correct! However, the rules in Android OS should remain the same. Considering the example above, there is a web-browser on the left and maps on the right. Both of these apps are running side-by-side, which means we have 2 Android Activities in the foreground, which have special rules to consider.
Activity State
In Android, your app uses something called Activities which is, more or less, the page you see at any given time on the screen. The Activity will perform slightly differently depending on the Window Lifecycle State.
- CREATED
- DESTROYED
- INITIALIZED
- RESUMED
- STARTED
- PAUSED
- SLEEP
To better support Multi-Window apps we really care about SLEEP, and RESUMED. In single window mode when the Activity is in the foreground it will be in the RESUMED state.
In multi-window mode displaying 2 Activities ONLY 1 will be in the RESUMED state.
This means the 2nd Activity will be in a SLEEP mode, which means the Activity is still running and consuming memory but is not executing on the main UI Thread or allocating as much memory as a RESUMED Activity. These rules are mainly determined by the developer and they choose to make the app do more work or less work in SLEEP.
In my experience when building mobile apps, if the app isn’t in the foreground I try to deallocate as much memory as I can to reduce my app’s footprint on the system. When the app triggers back to the RESUMED state I re-allocate the memory. This is no longer my recommendation for new form-factor devices that require Multi-Window support as a regularly used feature.
This problem only applies to Android 7 through Android 8 as there is new behavior rules added to Android 9+.
Android 9.0 – Multi-Resume
In Android 9.0, the Android Open Source Project started to support the new form factors being introduced by foldable devices using a feature called Multi-Resume.
Multi-Resume Docs
All top-visible activities in free-form windowing mode are resumed.
This contradicts our statement earlier about Multi-Window Support – yes Android fixed the problem in Android 9 which is great news for app developers. Now you can display multi-windows on your Android.
Device and have them in all in the RESUMED state. This behavior will also apply for any Android devices connected to an external monitor or dual-screen devices such as the Surface Duo.
Risks and Considerations
Mobile App developers already have a very hard job making sure their software works on a plethora of devices and OS versions, especially in the Android Ecosystem. Multi-Resume introduces hardware risks that now need to be built into the App Code that weren’t there in previous versions.
Consider you are building a Camera App and the user decides to try and use the default camera app and your new camera app at the same time. You may have 2 hardware connected cameras, so the 2 apps could technically run at the same time, but there is a high chance of conflict. Any app that integrates with hardware should check if it is in use prior to executing as a precaution.
Android 7-8 and Android 9+
We mentioned earlier that Activity Lifecycle States are different in Android 7-8 and Android 9+.
- Android 7-8: Only 1 Activity is RESUMED
- Android 9+: All top-level Activities are RESUMED
My recommendation is going to be adding specific code to support Android 7-8 vs Android 9.
Android 7-8 will be the versions that cause you the most headache’s because you want your app to still be running while in SLEEP mode. Since I haven’t built any Multi-Window apps I am not sure of the side effects, but it is something to be aware of when building your app for Android 7-8.
Android 9+ completely removes this problem as far as I can tell from the documentation available today. This means I don’t need to worry about handling app state as it changes since all top-level Activities will remain in RESUMED state.
The UI Thread
I was recently discussing these topics with my good friend Geoffrey Huntley and he asked me the hardest question.
Is there 1 UI Thread or more than 1 UI Thread?
The short answer is, I really have no idea. After reading the docs I have some opinions.
In Android 7 – 8 I would argue there is only 1 UI Thread, there can only be 1 Activity that is in the RESUMED state. Even in Multi-Window mode the focused Activity is RESUMED, the other activities are in SLEEP. This means there really is only 1 UI Thread and when the user switches between activities the UI Thread switches to that Activity. (Disclaimer: this is a theory and may not be the exact way it works).
In Android 9+ Multi-Resume is introduced, and this leads me to believe there are multiple UI Threads. If all top-level visible Activities act in the RESUMED state, there must be a UI Thread for each Activity that is visible. There could be something built into the Android OS that simulates multiple UI Threads, as I am not 100% sure. (Disclaimer: this is a theory and may not be the exact way it works)
Emulator Support
If you go and try creating a Dual-Screen Emulator you are going to struggle because there aren’t any base images to support this yet. Samsung has been hard at work making it easier to develop apps for the Galaxy Fold and handle their specific form factor. Check out the documentation here.
To get started:
- Create a Tablet – Nexus 10 Emulator
- Install the FoldableEmulator.apk
Once you install the emulator you will have quick form factor buttons to switch the Emulator between opened and folded.
Surface Duo Emulation
As of right now there is nothing available to emulate the Surface Duo or recommendations from Microsoft. After analyzing the Android Developer Documentation, I can simulate the Android behavior by using Multi-Window mode on a large tablet.
My recommendation to simulate Surface Duo:
- Create an Android Emulator with the Nexus 10 base image and make it a large tablet
- Open any app such as the web browser
- After you launch your app turn on multi-window mode
By leveraging the Multi-Window mode during development on a tablet emulator you are effectively simulating the same rules Android OS will execute on the Surface Duo running apps on different screens. (Disclaimer: Microsoft may be making changes to the base Android OS which may change Multi-Resume).
Beware: These are early recommendations and this guidance is not in any way an official recommendation from Microsoft. This is a way to get started early prior to Microsoft releasing the official emulators and documentation.
Xamarin.Forms or Uno Platform
I prefer to develop apps using .NET Technologies using Model-View-ViewModel Architecture which is very popular in both Xamarin.Forms and Uno Platform. Most of my app development techniques really won’t change for the Surface Duo or even the Galaxy Fold.
My favorite part of using these platforms is the ability to hook into native APIs and control your app exactly as you would if you were writing it natively in java or kotlin.
The important thing is to be aware of the potential for problems and how you may solve some of those problems.
You don’t want to spend ¾ of a release and realize that you need to re-write your entire App Sleep routine because one of your devices is on Android 8 which doesn’t support Multi-Resume.
Disclaimer: Observations from Documentation
This entire article is filled with observations I have made from Android documentation and does not provide any official recommendation from Microsoft for APIs that may or may not be exposed to make app development easier. Microsoft’s statement has been developer APIs will not be available for a few months.
