A quest to unify web games and stand alone development
Flash is the most popular game platform on the web. OpenGL in C++ is the most widespread fully accelerated gaming platform for downloads. Is it possible to target both platforms from the one code base? I think so...
-
Some Help Getting Started With Hxcpp
Tony at touchmypixel.com has posted some very useful information to help you get started with hxcpp. Be sure to go over there and check it out.
-
Haxe3D C++ Demo.
Nicolas Cannasse has written a 3D engine for flash 10. I have ported this code to neko and c++ using the neash library.
To test the code, first extract the contents of the zip and then look in the bin directory. Double click the h3d.exe or h3d_ogl.bat files to run the c++ version (the h3d_ogl.bat uses opengl, the other version uses the software renderer). You can see the flash version from the .html file (note: you will need flash player 10). You can run the neko if you have the haxelib version of nme installed by running “neko h3d.n”. All the source files are there in case you want to do some mods.
I have changed the mouse-look scheme from the original code to keep things a bit more centred. Move the mouse left/right to spin the objects, move it up/down to raise or lower the camera. You can use ‘w’ key for wireframe and the ‘p’ key for shading effects.
On my faster computer, I get about 100fps for the opengl renderer, 80fps for the software renderer, 40fps for the flash10 renderer, and 1fps for the neko/software renderer. There are some optimisations that could be done for the neash API, but really significant improvements could be gained by moving some of the engine into c++ code.
Currently, it is windows only. You can get it here.
Update:
There were reports of preformance loss in opengl, and I think I may have traced it to excessive texture transactions due to the text in the status panel. To work around this, I have cached the text as glyph bitmaps (doing this was on my todo list) which seems to have helped. (This update is in the svn version of neash). I have also added a command-line switch to use a simple fps counter instead of the status panel. You can try this out with the h3d_ogl_simple.bat command, In the new download.
-
Hxcpp 0.4, NME 0.9, Neash 0.9 Released!
What the flash?
What is Hxcpp? Hxcpp is the c++ backend for haxe. This means you can compile haxe code to c++ code, and then compile this to a native executable, for Windows, Linux or Mac.
What is NME? NME is the “Neko Media” library that wraps SDL, providing gaming interfaces for neko, and now native compiled haxe code.
What is Neash? Neash is a compatability layer that presents the flash API to haxe code running on other systems, such as js, neko or c++ native code.
Together these allows you to write code to target flash SWF files, and also cross compile to native code for Windows, Linux or Mac.
Hxcpp on haxelib
I have finally packaged up a bunch of changes into offical haxelib releases. Hxcpp is now on haxelib, which means you can get it with “haxelib install hxcpp”. This effectively creates a whole separate install of haxe, which can be run side-by-side so you can test it out without risk.
The cpp backend now supports Mac(intel) and Linux as well as the original Windows platform.
The main change to hxcpp is the packaging - moving towards a the final installation form. Currently there are a whole bunch of files distibuted in this release that should become redundant once the c++ backend is merged into the main branch. Also, the library coverage has been expanded a bit, but it is still not complete.
Usage
Firstly, you will need to run “haxecpp” instead of “haxe”. This executable is found in the appropriate bin subdirectory. I’m not sure if the “executable” flag will survive the compression, so you may need to “chmod a+x” the file.
It is probably best to place the appropriate bin directory in your executable path. On windows, this will also solve the problem finding the dynamic link library, hxcpp.dll. And on all systems, this will allow you to use the “make_cpp” command from the hxml files. On Linux systems, you will have to allow the executable to find the hxcpp.dso. This is most easily done by setting LD_LIBRARY_PATH to the bin/Linux directory, or copying this file into an existing library path. Similarly on Mac, you should set DYLD_LIBRARY_PATH.
To build haxe code, use “haxecpp” inplace of “haxe”, with a target specified by “-cpp directory”. This will place source code and a makefile in the given directory. Then you need to do a “make” on linux/Mac, or “nmake” on Windows to build the executable. You may need to set the environment variable “HXCPP” to point the the directory that contains this file. On windows, this will be something like: c:\Progra~1\Motion-Twin\haxe\lib\hxcpp\0,4\
As a shortcut, if you are using a hxml file, you can use “-cmd make_cpp” which will do the build for you assuming you used the “-cpp cpp” directory.
Neash/NME
The big changes for NME is that it now supports Linux and Mac(intel) for neko ac c++ targets. There have been a few bug fixes as well as a few new features:
- Bitmap class
- Expanded and optimised TileRenderer for render scaled and rotated sub-rects from a surface
- A few smarts for finding fonts, if no ttf is supplied
- Some blend modes have been added
- Added scale9Rect
- Added drawTriangles, with perspective correct textures
ToDo
There is still plenty to do, including, but not limited to:
Hxcpp:
- Proper coverage of all APIs.
- Resolve the order-of-operation problem: In c++ f(x++,x++) is ambiguous as to what order the increments are performed. Or perhaps agree to live with it.
NME:
- Add all blend modes
- Add all filters
- Discuss with experts the merits of static vs dynamic linking Mac and Linux.
Neash:
- Sound is a big ommision
- Loader code
- Unit testing of supported APIs.
Despite these issues, I think there is a useful core of functionality here.
Let me know what you think.
-
Haxe Preloader For Flash – Written in Haxe.
There has been some talk about creating flash preloaders for haxe. However, these methods step outside the haxe toolchain and add some additional complication.
I have come up with a reasonably simple method for creating a haxe preloader in haxe, and then linking to a (very almost) unmodified swf generated in haxe using a small neko program to produce a single file. The neko program uses code from the hxformat project, some of which is provided so you can easily recompile the tool.
Since each original haxe swf contains one frame, the resulting code contains 2 frames. The first frame contains the preloader which waits for the whole file to load and then locates the PreloaderBoot class by name. This class runs the appropriate initialisation code, creating and running the correct “main” class.
For classes that appear in both preloader and the main swf, flash takes the first one – the one form the preloader. This means that both classes will have the same “flash.Lib.current” and (almost) everything will just work.
One complication comes from the fact that the flash.Boot class is given a unique name for each of the swfs. This means flash.Boot class in the main swf is not automatically “new”ed and placed on the stage, and the standard haxe initialisation code is not run. To compensate, we manually set the trace function and call the Boot initialisation code explicitly.
This sounds a little dodgy to me, but it seems to work – I will have to do some more testing.
The “Main” class in the example zip contains a resource to pad it out. The preloaded swf can be seen on it’s own screen. You can refresh to see it loading.
The example code is in haxe-preloader-0.1.zip
All code & data there is public domain, except for the hxformat code, which has its own license. Use at your own risk.
-
HXCPP 0.3 Released
I have put together a new version of hxcpp, the c++ backend for haxe. New features include improved coverage of language features. All the unit tests except remoting pass now. I have also cleaned up the ocaml code a bit and improved the output consistency. Still a bit to do here, but not that much.
The code now contains a dependency system that allows for incremental compiling, greatly improving the speed.
Dlls are now all in one directory – by adding this to the exe path no dll copying should be required. This still needs a little thought :- I tried to delay load the dll, thereby giving greater control over locating it, but it seems the rtti system brings the dll in before I can change the load path. Apart from this, most of the 0.1 TODO list is finished. There are still a few little language features required – such as “break” from within a return block but it’s 99% there, and the external libraries are pretty much untested.
I have also hijacked the neko code to provide OS libraries. This means that I had to allow “neko” class paths in the cpp target – this seems a little odd – I will have to think about a solution here.
Strings are implemented with wchar_t, rather than utf8 bytes, so some neko functions that took “string” actually take “byte array” in cpp. On the plus side, multi-byte characters are “native” in the c++ target.
The source and demos are in hxcpp-03.zip.
-
HXCPP 0.2 – Huge performance increase.
I have switched hxcpp over from using ref-counting to using Boehm garbage collection. I have also added some additional perfromance improvements, such as integer-index field names to make interaction with neko more efficient.
The overall result is that for the Physaxe demo, the frame rate went from 24 fps to 82 fps (in opengl mode). The swf file runs at about 35 fps, and neko at about 8 fps. This is about what I was hoping for from the first round, but I got there in the end.
You can download the updated files here.
-
Boehm GC, virtual inheritance and finalizers.
I’m trying to get a speedup for the cpp backend for haxe by using garbage collection. Initial results are very promising – potentially about twice as fast. Howerver I spent a good few hours getting the the bottom of a little problem. Boehm garbage collection is a very impressive piece of work – it has all sorts of magic that does magical things, such as deal with virtual inheritance. This was a bit of a surprise because you do not always get “real” pointers, when you store an object pointer, you get one with an offset. However it seemed to work. Until I added finalizers to the external draw objectes used by the renderer. Apparently, you can only add finalizers to the “real” pointers (ie, those returned from “GC_MALLOC” et al), rather than a pointer to the same object related by virtual inheritance. The symptom was that the object gets finalized in the first “gc_collect”, even though it was still “used” as far as I was concerned. I guess this is not too surpising, and the fix was pretty easy, but the fact that everything else worked so well lulled be into not suspecting this initially.
-
C++ backend for haXe
I have just completed an alpha release of a c++ backend for haxe. This means that you can complile haxe code into a 100% compiled executable. You can download the demo file in hxcpp-01.zip. Sorry, windows only at this stage.
The distribution contains a new cpp backend for haxe. It has been based on a 2.0 version of haxe, which may be a tiny bit out of date. Most of the changes are in the new “gencpp.ml”, and to the standard library files, with a few little extra bits here and there. You can re-compile the haxe compiler if you have ocaml by using the supplied install.ml script.
To try this version for yourself, first backup your haxe distro and copy then supplied “compile/bin/haxe.exe” and “compiler/std/*” files over the top. Use the “-cpp cpp_directory” command line to generate a directory that contains src, include and nmake files. You can then compile these using the microsoft visual studio “nmake” utility. The build system requires the library, include, make and dlls from the “hxcpp” directory. To access these, you should set the environment variable “HXCPP” to point to hxcpp directory extracted from this distribution. This can be done from right-click-”My Computer”/Properties/Advanced/Environment Variables, or from the commandline before compiling. These resulting “exe” file also needs the hxcpp.dll file from the hxcpp/dll directory. The should be in your “path”, or simply copy it next to your exe.
You can recompile the hxcpp.dll using the nmake file in the directory. You can change the compile flags from the $HXCPP/nmake.setup file (eg, turn on debug).
Demos
Two demos have been included – “perf”, a small benchmark program I found on the net and a “Physaxe” demo. The source is included (slightly modified), and so are the binaries. The cpp src and include directories have been included to give you taste of the output if you can’t be bothered setting up the compiler yourself. The binaries can be found in demos/bin, and are compiled for neko, swf and cpp. The neko version can be run with “neko phx.n” or “neko TestRunner.n”. You do not need a very recent version of neko, but you do need the included “nme.ndll” findable by neko (next to it will work).
The cpp version of Physaxe uses the cpp verion of NME. This was compiled from the same code base as the neko version, except it uses the “neko.h” file found in the hxcpp directoty instead of the one that comes with neko. The nme.dll should be next to the compiled exe.
If you want to compile the nme versions yourself, you will need the latest nme and neash versions from code.google.com:
http://code.google.com/p/nekonme/source/checkout
http://code.google.com/p/neash/source/checkout
Performance
The flash version of physaxe runs the fastest, with the cpp version about 70% of the speed (when using the opengl version), and neko about 20% of the speed.
One of the problems is that the cpp version uses the neko api, which required fields to be looked up by name, which is quite slow in this implementation. A faster version could link directly to the hxcpp objects – but then it could not use the same API. This problem is made far worse by the fact the physaxe re-renders each point in each object every frame, rather then simply adjusting the matrix of existing objects.
I think the most significant loss of perfromance is coming from the reference counting housekeeping. I will look into a garbage collected system soon.
The results from the “TestRunner” are mixed with flash being faster for stings, but cpp faster for maths and looping. Neko is fastest for the sting sort in this case, but this is unusual because the stings are already sorted. When they are not, neko is very slow. The cpp string code is very simple, so there is scope for improvement there.
TODO
There is still plenty to do.
- A lot of the operators (eg, “*=”) have not been looked at.
- The actual formatting of the generated code needs a complete overhaul.
- The ml code needs some simplifying/cleaning.
- The standard libraries (eg, xml,regex)
- Need some way of locating the various dlls etc.
- Splitup/refactor the HObject.h et al files.
- Returning values from blocks/swithes.
- Complete neko.h
- Look at GC.
Plenty more, I’m sure.
-
Neash/NME 0.8 released
While this blog may have been quiet, I have been busy. The next version of Neash and NME has been released on haxelib, making it very easy to upgrade. Some cool features include:
- Fonts
- Filters (some)
- Bitmap Caching
- Improved opengl speed
- SWF reading/playing
- Scroll rects (axis aligned)
- Haxe 2.0 upgrade
- Getting the libraries. Simply use the “haxelib” tool that comes with haxe to download and install the “NME” and “Neash” libraries. From the command (shell) prompt, type: haxelib install nme followed by haxelib install neash
- Create the compiler command. Rather that typing haxe -main …… every time, you can create a “.hxml” file that contains the commands, then you can simply use haxe file.hxml. The hxml file contains flags or key-value pairs of command-line arguements. You can also use the “–next” to compile to more than one target from the single invovation of haxe.
To use neash, you will need the nme and neash libraries. To add these, you can use the command-line options “-lib neash” and “-lib nme”. For the neko target, you will also need to redirect the “flash” code to use “neash” instead. This is easily done with the “–remap flash:neash” command.
So a hxml file that targets both flash and neko looks something like this:
-main Simple -swf Simple.swf -swf-version 9 -swf-header 640:480:100:334433 -lib neash -cmd echo SWF done --next -main Simple -neko Simple.n --remap flash:neash -lib nme -lib neash -cmd echo Neko done
- Source code mod. You need to do some very minor modifications to run the neko version using neash. Specifically, you need to call “Init” and “Run” and the first and last things you do in your main routine. eg:
static public function main() { neash.Lib.Init("Simple",640,480); neash.Lib.SetBackgroundColour(0x334433); new Simple(); neash.Lib.Run(); }
But perhaps an introduction is in order. The purpose of Neash is to allow you to create programs that run on both flash and also natively, say as a downloadable program. You start with a simple (or complex) haxe program, targetting flash.
import flash.display.Sprite;
import flash.display.Shape;
class Simple extends Sprite {
public function new() { super(); flash.Lib.current.addChild(this);
// creating a new shape instance var circle:Shape = new Shape( ); // starting color filling circle.graphics.beginFill( 0xff9933 , 1 ); // drawing circle circle.graphics.drawCircle( 0 , 0 , 40 ); // repositioning shape circle.x = 80; circle.y = 80;
// adding displayobject to the display list addChild( circle ); }
static public function main() { new Simple(); } }
And you get a SWF that renders something like this.
This works well in a browser, but what if you wanted to distrubute this as a stand-alone exe? You would of course use one of the may flash-to-exe tools around. These all, one way or another, involve packaging up the flash runtime, and this has licensing implications. Also, it can be difficult to add DRM or other native extensions to the code. So the alternative offered here is to compile it to neko!
There are 3 simple steps for compiling to neko. 1. Get the libraries, 2. create the compiler command-line and 3. some very minor source code mods.
So running haxe on this hxml file will produce both “Simple.swf” and a “Simple.n” files. You can run neko Simple.n to run the neko program, producing much the same result. You can use neko Simple.n -opengl to run with opengl acceleration – although that will no be much use in the simple case.
All these project files, along with many others, can be found in the “samples” area of the neash library that you get when you use haxelib to install neash.