Handmade Hero » Forums » Code » Guide - How to build a native android app using a single batch file for Visual Studio
HawYeah
14 posts
#14668 Guide - How to build a native android app using a single batch file for Visual Studio
1 month ago Edited by on March 27, 2018, 4:15 p.m.

Hey guys, I spent the last couple of days trying to iron out how to build my game for android and I figured I'd explain how I did this to save the trouble for everyone else. So first, you want to download visual studio 2017 and when you begin installing it, make sure you check mobile/cross-platform c++ development. We want visual studio to download the android/java tools we will need to build native android apps as well as use visual studio as our debugger for android. To make sure everything went well, create a new project and make it a c++ android native-activity application. Run this application on some emulator or device and you should get flashing colors in your emulator/device.

// WINDOWS 7 ONLY

If you are running windows 7, the program will hang saying waiting for emulator. This is because the default android emulators in visual studio require windows 8.1 or higher. To fix this, you can either run the program on a android device that you own, or go to C:\Program Files (x86)\Android\android-sdk and run SDK manager as administrator. There, you can download a emulator for android 21 (it doesn't particularly matter which one u download). Once you downloaded it, you create a new avd for it by running AVD Manager in the same folder. Once AVD Manager opens up, go to device definitions tab, select some already existing device (eg. Nexus 10), and press create device. You will want to set Target to android 21, and set the CPU/ABI to the one that you downloaded from the SDK Manager. Check mark Use Host GPU to speed up emulation and hit ok. You can now use this emulator instead of the preinstalled emulators that came with visual studio and it should run on windows 7.

// END WINDOWS 7 ONLY

Now that we know that our android tools are working, we will want to setup our sample program. I have my folders setup similarly to the way handmade hero sets up its code. I have a build_android folder, a code folder, data folder, etc. My android manifest is stored in the code folder while my \res\values\strings.xml is stored in my data folder. I'm not going to be going over what these files are or how android native code works but if you want to know, you can look find a better tutorial at http://www.ikerhurtado.com/androi...ty-app-glue-lib-lifecycle-threads

The code for this guide is the same code that visual studio provides for the sample android native-activity (android_native_app_glue.h/c, main.cpp, AndroidManifest.xml, strings.xml), so open up a sample android native activity and copy those files into your code directory. Your directories should look like this:

android_project/build_android
android_project/code/android_native_app_glue.h
android_project/code/android_native_app_glue.c
android_project/code/main.cpp
android_project/code/AndroidManifest.xml
android_project/data/res/values/strings.xml

Okay, so we have all the important files for our native activity, now we need to build it. Thankfully, visual studio comes with everything you need to properly build your android programs. The android build process works something like this:

1) Build dynamic libraries for each of the CPU instruction sets we are targeting
2) Create an apk file with an AndroidManifest.xml, our applications resource files, and our dynamic libraries.
3) Sign our apk
4) Align our apk

1) To build your dynamic library, you have 2 options, you can use the ndk build system to generate the dynamic libraries (.so) for you, or you can just call clang provided by ndk directly to build the dynamic libraries (we need ndk to have access to headers and libraries but we don't need to use the ndk build system). In this tutorial, we will be using clang directly. When building for android, we need to be able to include the OS headers that our program may use, and we need to link against the OS dynamic libraries to actually call the functions that we use from the headers. Here is a list of the different includes we need to have, as well as their file locations

C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\sources\android\support\include
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\sources\cxx-stl\llvm-libc++\include
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\sources\cxx-stl\llvm-libc++abi\include

This includes a bunch of C/C++ standard library headers

C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\toolchains\llvm\prebuilt\windows-x86_64\lib64\clang\5.0.300080\include

These are the clang headers, they provide you with CPU intrinsics (simd) as well as other clang compiler specific intrinsics.

C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-arm\usr\include
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-x86_64\usr\include
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-x86\usr\include
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-arm64\usr\include

These are the OS headers, you only include one of these directories depending on which OS version and CPU you are targeting. I listed out 4 different cpu archs but for this tutorial, I will be using the x86 directory. Visual studio provides directories for some CPU archs and recent versions of android but if you need something that isn't provided by default, you can download it through the Android SDK Manager (Visual studio installs it at C:\Program Files (x86)\Android\android-sdk).

Now with those headers, we can compile our C\C++ files into valid object files using clang. Clang is also provided by NDK and can be found in C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe. Older versions of NDK used GCC but I imagine that the process for doing this would be something similar. I won't be going over how to use clang since I'm not particularily knowledgeable, but these two commands in your batch file will compile the android_native_glue.c as well as your own main.cpp:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
set NDK=C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c
set CLANG_NDK=%NDK%\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe

set CompilerFlags=-I "%NDK%\\sources\\android\\support\\include" 
set CompilerFlags=-I "%NDK%\\sources\\cxx-stl\\llvm-libc++\\include" %CompilerFlags%
set CompilerFlags=-I "%NDK%\\sources\\cxx-stl\\llvm-libc++abi\\include" %CompilerFlags%
set CompilerFlags=-I "%NDK%\\toolchains\\llvm\\prebuilt\\windows-x86_64\\lib64\\clang\\5.0.300080\\include" %CompilerFlags%
set CompilerFlags=-g2 -gdwarf-2 -Wall -O0 -x c++ %CompilerFlags%

set x86CommonFlags=-fdiagnostics-format=msvc -target "i686-none-linux-androideabi"
set x86CommonFlags=-gcc-toolchain "%NDK%\\toolchains\\x86-4.9\\prebuilt\\windows-x86_64" %x86CommonFlags%
set x86CommonFlags=--sysroot="%NDK%\\platforms\\android-21\\arch-x86" %x86CommonFlags% 

set x86CompilerFlags=-I "%NDK%\\platforms\\android-21\\arch-x86\\usr\\include"

call %CLANG_NDK% -c -o "%OutputDir%\\android_native_app_glue.o" %CodeDir%\\android_native_app_glue.c %x86CommonFlags% %x86CompilerFlags% %CompilerFlags% 
call %CLANG_NDK% -c -o "%OutputDir%\\main.o" %CodeDir%\\main.cpp %x86CommonFlags% %x86CompilerFlags% %CompilerFlags%


So there are a couple of things to notice. First, we specify what CPU our code will run on using the -target flag (we specify it for compiling and linking). In this case, I was compiling for x86 but here are the targets for x86-64, Armeabi-v7a, and armeabi64-v8a:

x86_64-none-linux-android
armv7-none-linux-androideabi
aarch64-none-linux-androideabi

There are also two other important commands, the -gcc-toolchain and --sysroot. sysroot is a path to libraries like EGL, OpenGLES, while gcc toolchain I'm less sure about. From what I can tell, it tells clang how to output machine code in the object files but I'm not 100% sure about this. The sysroot path is different depending on which OS and CPU you are targeting so I listed a couple paths below:

C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-x86
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-x86_64
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-arm
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-arm64

The rest of the compiler commands are standard clang flags (for documentation on clang flags, you can find more at https://clang.llvm.org/docs/Clang...html#debug-information-generation). So now we have 2 object files that we need to link to create a dynamic library. Dynamic libraries on android are .so files, and you call clang again to link your object files into libraries. There are again a bunch of static libraries that we need to link with to get android and C/C++ standard library support so I'll list them below:

C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\platforms\android-21\arch-x86\usr\lib
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\toolchains\x86-4.9\prebuilt\windows-x86_64\lib\gcc\i686-linux-android\4.9.x
C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c\sources\cxx-stl\llvm-libc++\libs\x86

These are the android libs, cpu features lib and c++ lib. The clang command to link our .so files now looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
set LinkerFlags=-landroid_support -lc++_static -lc++abi -llog -landroid -lGLESv1_CM -lEGL
set LinkerFlags=-shared %OutputDir%\\android_native_app_glue.o %OutputDir%\\main.o %LinkerFlags%

set x86CommonFlags=-fdiagnostics-format=msvc -target "i686-none-linux-androideabi"
set x86CommonFlags=-gcc-toolchain "%NDK%\\toolchains\\x86-4.9\\prebuilt\\windows-x86_64" %x86CommonFlags%
set x86CommonFlags=--sysroot="%NDK%\\platforms\\android-21\\arch-x86" %x86CommonFlags% 

set x86LinkerFlags=-Wl,-rpath-link="%NDK%\\platforms\\android-21\\arch-x86\\usr\\lib" 
set x86LinkerFlags=-Wl,-L"%NDK%\\platforms\\android-21\\arch-x86\\usr\\lib" %x86LinkerFlags%
set x86LinkerFlags=-Wl,-L"%NDK%\\toolchains\\x86-4.9\\prebuilt\\windows-x86_64\\lib\\gcc\\i686-linux-android\\4.9.x" %x86LinkerFlags%
set x86LinkerFlags=-Wl,-L"%NDK%\\sources\\cxx-stl\\llvm-libc++\\libs\\x86" %x86LinkerFlags%

call %CLANG_NDK% -o"%ApkDir%\\lib\\x86\\libnative-activity.so" %x86LinkerFlags% %x86CommonFlags% %LinkerFlags%


The first thing you'll notice is that we tell clang which libraries we want to link against by using the -l flag. So for example, to link with the android library, we wrote -landroid. If you want to link with a new library, you'll have to add them in the same format. The other thing to notice is that we add a rpath-link. Clang has a runtime linker that looks for libraries at runtime to link with our program and rpath provides those libraries. And now we have a .so file thats compiled for android! There is a small caveat, we have to name our library properly. In the android manifest.xml, we provide the name of our native applications .so. The actual name of our .so has to be that same name but with a lib put at the beginning and .so at the end. So for example, if we name our library in the manifest HelloWorld, then the actual file name needs to be libHelloWorld.so. This is a easy spot to make a hard to find error so keep this in mind.

2) So we now need to create an apk file and stuff it with all our data. The android sdk provides us with a program called aapt that lets us do this. The directory within which aapt is installed is %AndroidCmdDir% and below is an example call to aapt:

1
2
3
4
set OutputDir=..\build_android
set ApkDir=%OutputDir%\apk

call "%AndroidCmdDir%\aapt" package --debug-mode -f -M %CodeDir%\AndroidManifest.xml -S ..\data\res -I "%AndroidDir%/platforms/android-21/android.jar" -F AndroidTest.unaligned.apk %ApkDir%


You can see that we pass it the android manifest xml as well as our resource file directory. We also provide a jar file for the android os we are targeting (jar files are just a bunch of java classes put together that acts like a library), and we specify that this apk is a debug mode apk which lets us debug it in Visual studio. We also provide a directory at the end (%ApkDir%) which tells aapt that whatever is in this directory, copy all the folders and folder contents to the apk. This is important because Android expects the apk folder layout to look a certain way. So for example, our application is native so when our app is launched, android needs to find the .so file that is the main program entry. It also has to make sure that it gets the .so for the CPU that the program is being run on. We can't have it try to run the arm version on a x86 machine for example. So the file structure that android expects is, that we have a folder called lib and within it, we have subfolders for each CPU architecture our app is targeting. Inside those individual folders is the .so file for that specific CPU arch. So if we are compiling for x86, arm, and arm64, we would have something like this:

lib\x86\libnative-activity.so
lib\armeabi-v7a\libnative-activity.so
lib\arm64-v8a\libnative-activity.so

If you want a more full documentation on apk file structure, you can read more at this link (http://www.ryantzj.com/android-ap...package-apk-structure-part-1.html). Okay so with our folders being arranged as android expects them, we provide the directory with this folder layout to the end of aapt so that aapt just copies that structure into the apk. If your ever wanna check if your apk is structured correctly, you can always use WinRar or some other zip program to unzip and check its contents. This is because apk's are just special zip files so they will be read correctly by any zip program you have. For a more comprehensive doc on aapt, go to https://elinux.org/Android_aapt.

3) Now we have an apk with all of our libraries and xml files in it and we need to sign it for android to consider our apk a valid one that it can install (and to secure it). So we have to create a keystore. Keystores are not to be created on every compilation. You usually want to create a keystore once and just use it all the time afterwards to sign your apk. To create a keystore you use the keytool provided by the android sdk (C:\Program Files(x86)\Android\android-sdk\build-tools\25.0.3\bin\keytool.exe):

1
call keytool -genkey -keystore %OutputDir%\debug.keystore -storepass android -alias androiddebugkey -dname "CN=company name, OU=0, O=Dream, L=CityName, S=StateProvince, C=Country" -keyalg RSA -keysize 2048 -validity 20000


When you call this, you will get a prompt asking you to enter key password. If you just hit enter, the key password will be the same as the store password which is okay for our debug app. I put in dummy names but for your application, you should populate the keystore with valid data (put your company name, your city, state, country, etc). You can play around with different encryption methods for your application but for this tutorial, a straight copy of what I provided should work. For more info on the keytool, check out https://docs.oracle.com/javase/8/...technotes/tools/unix/keytool.html.

Now that we have a keystore, we can use either jarsigner or apksigner to sign our apk (they can be found at C:\Program Files(x86)\Java\jdk1.8.0_131\bin\jarsigner.exe, C:\Program Files (x86)\Android\android-sdk\build-tools\25.0.3\apksigner.bat respectively). From what I read online, apksigner has more features and is the preferred way to sign apk's but I also had a bug where it was deleting my apk's so currently I use jarsigner. You can try either or, for making a debug app it shouldn't really matter. Once you have something you want to release, you should probably then see what kind of signing options you want on your app for maximum security. So to sign our apk with jarsigner, we use the below call

1
call jarsigner -sigalg SHA1withRSA -digestalg SHA1 -storepass android -keypass android -keystore %OutputDir%\debug.keystore -signedjar AndroidTest.signed.apk AndroidTest.unaligned.apk androiddebugkey


You will notice that we have to provide the keypass and storepass that we used for the keystore in order for the signing to work correctly. For more info on the apksigner and jarsigner, you can find more at https://developer.android.com/studio/command-line/apksigner.html and https://docs.oracle.com/javase/7/...otes/tools/windows/jarsigner.html respectively.

4) Lastly, we want to align the contents of our apk to 4 byte boundaries. This will let android read them faster during runtime. I don't think this step is required but its standard for android. We do this by calling zipalign which can be found at C:\Program Files(x86)\Android\android-sdk\build-tools\25.0.3\zipalign.exe.

1
call "%AndroidCmdDir%\zipalign" 4 %OutputDir%\AndroidTest.signed.apk %OutputDir%\AndroidTest.apk


For more on zipalign, go to https://developer.android.com/studio/command-line/zipalign.html. Now we have a apk we can run and install on our devices!! Visual studio lets us debug already built apk's by opening them up as projects and telling visual studio where our symbols are. The thing is, unlike visual studio, clang puts the symbols inside of the dynamic library. There are tools that you can use to separate them but visual studio will be able to read the symbols from a .so without any problems. So after you opened up your apk as a solution in visual studio, go to Project, then hit Properties. In the window that pops up, in the additional symbol search paths, put the directories to where each of your .so files reside. Visual studio won't traverse sub folders to find it so you have to put a entry for every CPU target you are compiling for. You can find more at https://blogs.msdn.microsoft.com/...-android-with-visual-studio-2015/. Now you can run your application on your phone or emulator and debug it. The actual debugging step is a lot more janky than debugging regular win32 programs and visual studio tells you that sometimes it can miss a breakpoint, but as far as I'm aware, all android debugging is unfortunately like this. Below is the build.bat that I used to build this app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
set CodeDir=..\code
set OutputDir=..\build_android
set ApkDir=%OutputDir%\apk

set AndroidDir=%ProgramFiles(x86)%\Android\android-sdk
set AndroidCmdDir=%AndroidDir%\build-tools\25.0.3
set JAVA_HOME=%ProgramFiles(x86)%\Java
set NDK=C:\ProgramData\Microsoft\AndroidNDK64\android-ndk-r15c
set CLANG_NDK=%NDK%\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe

IF NOT EXIST %OutputDir% mkdir %OutputDir%
IF NOT EXIST %ApkDir%\\lib\\x86 mkdir %ApkDir%\\lib\\x86

pushd %OutputDir%
del *.apk >NUL 2> NUL

set CompilerFlags=-I "%NDK%\\sources\\android\\support\\include" 
set CompilerFlags=-I "%NDK%\\sources\\cxx-stl\\llvm-libc++\\include" %CompilerFlags%
set CompilerFlags=-I "%NDK%\\sources\\cxx-stl\\llvm-libc++abi\\include" %CompilerFlags%
set CompilerFlags=-I "%NDK%\\toolchains\\llvm\\prebuilt\\windows-x86_64\\lib64\\clang\\5.0.300080\\include" %CompilerFlags%
set CompilerFlags=-g2 -gdwarf-2 -Wall -O0 -x c++ %CompilerFlags%

set LinkerFlags=-landroid_support -lc++_static -lc++abi -llog -landroid -lGLESv1_CM -lEGL
set LinkerFlags=-shared %OutputDir%\\android_native_app_glue.o %OutputDir%\\main.o %LinkerFlags%

set x86CommonFlags=-fdiagnostics-format=msvc -target "i686-none-linux-androideabi"
set x86CommonFlags=-gcc-toolchain "%NDK%\\toolchains\\x86-4.9\\prebuilt\\windows-x86_64" %x86CommonFlags%
set x86CommonFlags=--sysroot="%NDK%\\platforms\\android-21\\arch-x86" %x86CommonFlags% 

set x86CompilerFlags=-I "%NDK%\\platforms\\android-21\\arch-x86\\usr\\include"

set x86LinkerFlags=-Wl,-rpath-link="%NDK%\\platforms\\android-21\\arch-x86\\usr\\lib" 
set x86LinkerFlags=-Wl,-L"%NDK%\\platforms\\android-21\\arch-x86\\usr\\lib" %x86LinkerFlags%
set x86LinkerFlags=-Wl,-L"%NDK%\\toolchains\\x86-4.9\\prebuilt\\windows-x86_64\\lib\\gcc\\i686-linux-android\\4.9.x" %x86LinkerFlags%
set x86LinkerFlags=-Wl,-L"%NDK%\\sources\\cxx-stl\\llvm-libc++\\libs\\x86" %x86LinkerFlags%

REM call %CLANG_NDK% -c -o "%OutputDir%\\android_native_app_glue.o" %CodeDir%\\android_native_app_glue.c %x86CommonFlags% %x86CompilerFlags% %CompilerFlags% 
REM call %CLANG_NDK% -c -o "%OutputDir%\\main.o" %CodeDir%\\main.cpp %x86CommonFlags% %x86CompilerFlags% %CompilerFlags%
REM call %CLANG_NDK% -o"%ApkDir%\\lib\\x86\\libnative-activity.so" %x86LinkerFlags% %x86CommonFlags% %LinkerFlags%


REM Keystore 
REM call "%JAVA_HOME%\jdk1.8.0_131\bin\keytool" -genkey -keystore %OutputDir%\debug.keystore -storepass android -alias androiddebugkey -dname "CN=company name, OU=0, O=Dream, L=CityName, S=StateProvince, C=Country" -keyalg RSA -keysize 2048 -validity 20000

REM Create APK file
call "%AndroidCmdDir%\aapt" package --debug-mode -f -M %CodeDir%\AndroidManifest.xml -S ..\data\res -I "%AndroidDir%/platforms/android-21/android.jar" -F AndroidTest.unaligned.apk %ApkDir%
call "%JAVA_HOME%\jdk1.8.0_131\bin\jarsigner" -sigalg SHA1withRSA -digestalg SHA1 -storepass android -keypass android -keystore %OutputDir%\debug.keystore -signedjar AndroidTest.signed.apk AndroidTest.unaligned.apk androiddebugkey
call "%AndroidCmdDir%\zipalign" 4 %OutputDir%\AndroidTest.signed.apk %OutputDir%\AndroidTest.apk


There is a problem still however with this setup. If we want to add a new CPU target to our app in the future, we won't know where to find the libraries and headers specific to that CPU target that we will need. So the way I found all of these directories is by making a native activity in visual studio (the sample program visual studio provides), and building it for all the different available targets. I had visual studio's output options set to verbose so in the output window in Visual studio, you would get a ton of text that you didn't before. If you search through it for ClCompile and ClLink, you should be able to find the clang.exe calls that visual studio does to build for that target. So you can copy those calls and see what directories visual studio uses and paste those in your batch file.

Hope this guide was helpful :).