Yeah loading executable code from sdcard (internal or external) won't work. That's an security issue so Android doesn't allow that.
There are few options you can try to do. First of all you don't need to do full apk build process. You need to run just the ndk-build to get shared library. Then put this library inside apk file (which is just an zip file), run zipalign and do debug code signing. Doing these last 3 steps manually instead of using SDK build tools will be significantly faster.
Another option is to try to put .so file to /data/local (if you are < Android v4) or /data/local/tmp (if you are >= Android v4) location. Typically everybody can write to these locations and dlopen allows to load shared libraries from this place. You might need to do chmod 777 on .so file you put there. It can happen that vendors like Samsung have modified Android and dlopen will still fail. And I believe Android v5 have changed /data/local/tmp - it also doesn't allow dlopen from that location. But writing files to that location (adb push) still works. Then you can do a bit more trickier workaround. First put .so file somewhere (/data/local or sdcard), and then use run-as helper to copy .so file to it's real location. Something like this:
adb shell "cat /data/local/tmp/libgame_lib.so | run-as com.your.package.name sh -c 'cat > /data/app/com.your.package.name-1/lib/arm/libgame_lib.so ; chmod 700 /data/app/com.your.package.name-1/lib/arm/libgame_lib.so'"
Just change com.your.package.name
to real package name (and "-1" suffix is version number you specified in manifest). It's been a while time since I have done this, so I may remember some thing wrongly. I think your package must have enabled debug attribute (in manifest) for run-as to work.
Another option would be to try to use adb backup command. It can download or upload full backups of applications on device (unless this is disabled in app manifest). You'll need to do backup to get package from device, unpack it (it is zlib compressed tar file), add your new files, pack it back and then restore on device with adb restore command. A bit more work is involved, but still should be faster than full SDK build.
Here's some blog post on both of these methods (run-as and backup): http://blog.shvetsov.com/2013/02/...ndroid-app-data-without-root.html