GitHub Documentation Patreon donate button

Stencyl (HaXe) Extension


  • administrator

    hey @mdotedot, are you going to open-source your Stencyl extension? I'd like to check the errors you're having, calling .leave() really shouldn't crash the application. Cheers!



  • If all goes well the Stencyl Extension will become available to the Stencyl Community.

    It would be awesome if you could help me with the crash-code.

    Maybe you know of an alternative way to do the match(Error.Custom(Error.Blocked) line.


  • administrator

    @mdotedot would you mind sharing your extension on GitHub? if you don't wanna make it public for now, you can invite me privately first maybe. Cheers



  • Steps to reproduce:

    Followed : https://www.wikihow.com/Install-Node.Js-on-Windows to install NPM

    set PATH=c:\"Program Files"\nodejs;%HAXE_PATH%;%NEKOPATH%;%PATH%
    node -v
    
    npm install typescript
    

    Download colyseus examples: https://github.com/colyseus/colyseus-examples : Clone/Download ZIP
    Extract in C:\

    cd \colyseus-examples-master
    npm install
    
    npm run bundle-colyseus-client
    npm start
    

    Now for the HaXe Part:

    Download https://github.com/HaxeFoundation/haxe/releases/download/3.4.4/haxe-3.4.4-win64.exe

    c:> haxe-3.4.4-win65.exe
    

    let it create default c:\HaxeToolkit :

    Setting DOS Environment:

    set HAXE_PATH=c:\HaxeToolkit\haxe
    set NEKOPATH=c:\HaxeToolkit\neko
    set PATH=%HAXE_PATH%;%NEKOPATH%;%PATH%
    
    haxelib setup c:\HaxeToolkit\haxe\lib
    haxelib --global update haxelib
    
    haxelib install openfl
    haxelib run openfl setup
    haxelib run lime setup 
    lime create HelloWorld
    cd HelloWorld
    lime test html5
    
    lime setup windows
    

    Download Visual Studio 16
    vs_community__956414624.1551197993.exe
    Choose : Desktop development with C++ AND Click C++/CLI support

    lime test windows
    

    The installed 4.0.8 hxcpp does not contain run script apparantly , so set to original hxcpp:

    haxelib set hxcpp hxcpp 
    

    When 'Error : Could not process asset libraries' then do:

    openfl rebuild tools
    

    COLYSEUS
    Download ZIP from GitHub : https://github.com/colyseus/colyseus-hx
    Extract the archive to c:\HaxeToolkit so that the project.xml is in c:\HaxeToolkit\colyseus-hx-master

    Modify the Main.hx: to mkae sure that the localhost line is active and the "ws://colyseus-examples.herokuapp.com"); is commented out

              this.client = new Client("ws://localhost:2567");
    

    Build the NyanCat example

    c:> cd \HaxeToolkit\colyseus-hx-master\example\NyanCat
    C:\HaxeToolkit\colyseus-hx-master\example\NyanCat>lime build project.xml html5
    

    Error: Could not find haxelib "haxe-ws", does it need to be installed?

    Install haxe-ws 1.0.5

    haxelib install haxe-ws 
    
    lime test html5
    

    It should now produce the nyancat window where you can move around with the cursor keys

    C:\HaxeToolkit\colyseus-hx-master\example\NyanCat>lime build project.xml windows
    
    lime test windows 
    

    LEAVING ROOM Crash:

    Now we change the Main.hx to include leaving room

    this.client = new Client("ws://localhost:2567");
    	//this.client = new Client("ws://colyseus-examples.herokuapp.com");
    this.room = this.client.join("state_handler");
    var timer = new haxe.Timer(5000); 
    timer.run = function() {
    	trace(" Leave Room ");
    	this.room.leave();
    	var timer2=new haxe.Timer(2000);
    	timer2.run = function (){
       	   timer2.stop();
    	   trace(" Create new room ... " );
    	   this.room = this.client.join("state_handler");
        };
    };
    

    Build and lime test the windows version

    When you see CLIENT: ERROR you should stop the program and start the program again.

    Repeat to use lime test windows

    After a while; approximate < 10 minutes; it will crash



  • After two weeks of further debugging I've now came up with a work around so that it does not crash.

    These are the modifications that I needed to make to avoid null pointer crashes:

    SocketSys.hx (haxe/net/impl/SocketSys.hx)

    					// M.E. the match error.custom(error.blocked) caused null pointer crash
    					/*
                        needClose = !(e == 'Blocking' || (Std.is(e, Error) && (
                            (e:Error).match(Error.Custom(Error.Blocked)) ||
                            (e:Error).match(Error.Blocked))
                        ));
    					*/
    					if((""+e).indexOf("Block") < 0 && (""+e).indexOf("Custom(EOF)") < 0){
    						trace('closing socket because of $e');
    						needClose=true;
    					}
    				}
    

    And further down I don't close the socket which caused blocking and crashing things on Android

    	if (needClose) {
    		//	    close();  // M.E. This will cause Android problems
    	}
    

    Room.hx : avoid null pointer exception

    private function patch( binaryPatch: Bytes ) {
            // apply patch
    		// M.E. Check for null on binaryPatch
    		if(binaryPatch == null || this == null || this._previousState == null){
    trace("BINARY PATCH IS NULL or this._previousState == null !");
    		}else{
    			 
    			this._previousState = FossilDelta.Apply(this._previousState, binaryPatch);
    			// trigger state callbacks
    			this.set( MsgPack.decode( this._previousState ) );
    			this.onStateChange(this.state);
    		}
            
            
        }
    

    FossilDelta.hx in Apply to avoid null pointer exception

    	var total=0;
    	// M.E. Check for null on src
    	  if(src == null)return haxe.io.Bytes.alloc(0);
    	  if(delta == null) return haxe.io.Bytes.alloc(0);
    	  // M.E. End of modification
    	  
    

    And I needed to inform Stencyl about errors in Room.onError or the extension would still use room functions ...

    Room.hx connect function

    	this.connection.onError = function (e) {
    			var message="Possible causes: room's onAuth() failed or maxClients has been reached.";
    			if(getGameAttribute("ColyseusErrorMessage") == null)setGameAttribute("ColyseusErrorMessage", ""); setGameAttribute("ColyseusErrorMessage", ""+getGameAttribute("ColyseusErrorMessage")+" "+message); 
                trace(message);
                this.onError(e);
            };
    

    In the application I created an onError routine and that would be hit sometimes but then it will do a new client instance and it starts creating rooms again.

    So we don't close the socket on haxe.net level but detect problems in Room and Client on Colyseus level.

    Hopefully these changes don't bite me later. I really want to continue working on other methods than Lobby mechanism!

    Mac OSX, iOS Simulator, iPhone 5, Windows and Android all work without crashes when creating and leaving rooms now.
    I ran a demo-game that left and created rooms every 10 seconds and they kept working for an hour.
    Android crashed on one device after an hour. It had done over 100 room creations by that time.
    I need to keep that in mind when doing more stuff later on and see if it reoccurs.
    (HTML5 never gave problems as it didn't use haxe.net)



  • This week I've made progress on a simple room where players can be joined and walk around.
    I made it so that players would be moved simultaneous to see if the server could handle it.

    It didn't quite run well with 2 or more browser sessions at the same time. Also a windows publication didn't fare well.

    Yesterday I found out that the CPU on Windows and memory on HTML5 where draining. So today I made a simple version on my NyanCat project. I had it running on both html5 and windows and it had the same effect.

    Then I stripped colyseus from the nyancat so that only the image was left with controlling keys and there was no problem with CPU.

    With Colyseus:
    alt text

    And the non-colyseus version:

    alt text

    So Haxe client is consuming a lot of CPU !

    At least it wasn't my client implementation since both have the same problem. I guess I need multiple computers from now on to do tests ....



  • Last days I've worked on refactoring code and CPU investigations.

    Connection.hx modifications to avoid CPU consumption

     #if sys
    		while(!this.isClosed){ //M.E. After 200 reconnections the CPU was gone to 92%
    			this.ws.process();
    				
    			 Sys.sleep(0.005); // M.E. Avoid CPU consumption by introducing a really small sleep
            }
     #end
     public function close () {
    		this.isClosed=true; //M.E. add for thread checking
            this.ws.close();
     }
    		
    

    Perhaps the this._enqueuedSend mechanism can be altered as well to check for the isClosed state?!?! But I'm not sure what the enqueued mechanism needs this...

    Running with over more than 200 client initialization tests (and client.closing them) the CPU consumption kept relatively steady.
    Now the Windows executable doesn't take much of the CPU.

    Remains Javascript that consumes a lot of memory. But this couldn't easily be reproduced on the HaxeKit version, so it might be something related to other parts of HaXe and Stencyl.

    I was able to switch rooms on Android, Windows and HTML5 though. I need to test the other targets with the latest build to make sure I didn't break anything for them.

    Current state:

    • Initialize Colyseus (tested 200x re-initializations)
    • Create Room (Lobby, State_Handler) : Also done together with the 200x initializations
    • Get room list (getAvailableRooms)
    • Join Room
    • Leave Room
    • Player list
    • Send Data
    • Receive Data

    TODO:

    • Investigate Javascript memory consumption
    • Publication to all Stencyl targets
    • Ping / latency mechanism
    • Broadcast investigations
    • onAuth investigations
    • Different Room Types : RawData,TurnBased,LockData,SimplePhysics,Chat


    • javascript memory consumption was a Stencyl bug. It has to do with drawing stuff, which I obviously did in my test application. Filed a PR on Stencyl Forum for this.
    • Publications:
      iPhone Simulator
      iPhone (Should be fine on iPad as well)
      Mac OSX
      Linux
      Windows
      Android (Tablet+Phone)
      HTML5

    So all is fine to continue working on the RoomTypes.



  • Hello! do you use haxe only for client code or you have created externs for server as well?



  • Hello Serjek,

    Currently I'm working on Stencyl Extension which uses (modified) HaXe Colyseus Client code.
    The server is still running on TypeScript. A HaXe server would be awesome to have but I currently have my hands full with Client Code !

    Kind regards from
    M.E.



  • This weekend I made:

    • PingTime
    • PlayerID from SessionID block
    • Player in Seat : This has to be changed to server-side, since it now apparently goes wrong sometimes (see pictures)

    Made a milestone today: LockRoomType

    This is one of the main goals of the Stencyl Extension.
    I have wanted a lock-mechanism behavior for a long time.
    The idea is that you have a grid-based game where the players 'lock' the part of the grid that will be changed.
    When a lock is granted the player can alter the grid (move/push/pull/alter state).
    Once data has been modified an unlock is send.

    What I have is still flawed by leaving/killing or anyother type of disconnection.
    When there is a lockstate it will not be unlocked.
    That will be altered by a time-exceeded-system so that the server will unlock the set of grid positions
    that the player/client had locked.
    I probably need to use a 'previous' data mechanism as well so that data can be reversed when it only partially has been changed.

    Made a prototype so that moves are done automatically.

    Current Progress:

    alt text

    Hmm... will need to double check system because running for over an hour it created another block :(

    alt text

    At least the player positions (the colorblocks) are in sync!



  • This week I've worked on :

    • Player in Seat - ServerSide
    • TurnBased RoomType: NextTurn, isTurn
    • MasterPlayerID for Client Side logic Room Type
    • Lock Mechanism

    The lock mechanism was flawed by not release locks and to not compare clientside grid with serverside grid.

    CLIENT
    For example when the client pushes a block from position 25 to 15 it will need to send 15=0,25=2,35=1000 to the server and request a lock on them:

    grid[15]=0; 	// free space
    grid[25]=2; 	// block
    grid[35]=1000; 	// playernr
    

    SERVER
    Server receives the lockrequest (15=0,25=2,35=1000)
    It checks his own grid if the positions match the client request lock. If the grids are out of sync it will send an error back.
    The client will then request a complete grid from the server as it is apparently behind.

    If the server grid position matches the requested lock positions a lock will be attempted: (pseudo-code)

    attemptposition=new Array();
    isOK=true;
    for each lockposition{
    	if(isOK && gridlock[lockposition] == 0){
    		gridlock[lockposition]=time; 
    		attemptposition.push(lockposition)
    	}else{
    		isOK=false;
    		// there is a lock on one of the 
    		for(each attemptpositon p) gridlock[p]=0;
    	}
    }
    

    When the server was able to lock all gridpositions it will send a success to the client

    CLIENT
    The client will do a grid request if the return state was an error.
    The client will send the changed data to the server when it was success, otherwise it will not do the move as there was/is a lock.

    SERVER
    When server receives the griddata it will set the new grid values and afterwards set the lockpositions to 0-time.

    After each request the server checks the gridlock table if the currenttime - locktime > 2 seconds .. when it finds one it will reset the lock.

    CLIENT
    Sends unlock to server.


    A room with four player-bots ran for over 4 hours and still they were in sync.


    Next I will need to figure out how to do classes with overloading / inheritence in TypeScript.
    I want all the room types to share code that are common in all rooms.

    For instance I need PlayerInSeat, PingTime, getPlayersInRoom etc.. code in all typescript rooms.



  • Subject: Colyseus HaXe Externs & Colyseus version 0.10

    Took a break from the Colyseus Extension, but I'm back.
    I was asked to create a MicroPhone HaXe extension for iOS and Android so that did take some time.

    Also, I wanted to wait a bit till the version 0.10 was done on both of the client and server side.

    Thanks to Serjek and Endel for their contributions, I was able to get a HaXe combination running with the Nyan Cat example.

    Server Side: Oracle Linux

    New Session

    • export LD_LIBRARY_PATH=/root/haxe/neko
    • /usr/local/bin/haxe
    • lix use haxe 4.0.0-rc.2
    • haxe -version
      4.0.0-preview.4+1e3e5e0
    • cd /root/serjek/colyseus-hxjs-examples-master
    • haxe client.hxml
    • cd bin/client
    • yarn
    • node index.js

    Test will run.

    Now for the HaXe Client on Windows: https://github.com/colyseus/colyseus-hx

    • https://github.com/HaxeFoundation/haxe/releases/download/3.4.4/haxe-3.4.4-win64.exe
    • set HAXE_PATH=c:\HaxeToolkit\haxe
    • set NEKOPATH=c:\HaxeToolkit\neko
    • set PATH=%HAXE_PATH%;%NEKOPATH%;%PATH%
    • haxelib setup c:\HaxeToolkit\haxe\lib
    • haxelib --global update haxelib
      Current version is 3.3.0
    • haxelib install openfl
      8.9.0
    • haxelib run openfl setup
      7.3.0
    • haxelib run lime setup
      Up to Date
    • lime create HelloWorld
    • cd HelloWorld
    • lime test html5
    • extract the colyseus hx master to \haxetoolkit\colyseus-hx-master
    • cd \HaxeToolkit\colyseus-hx-master\example\openfl\Source
    • notepad Main.hx
      Edit: the ws connection so that it points to the Linux Server mentioned above.
    • cd \HaxeToolkit\colyseus-hx-master\example\openfl
    • haxelib install haxe-ws
    • haxelib set hxcpp 3.2.94
      (installing)
    • lime build project.xml html5
      Error:
    Source/Main.hx:55: characters 48-50 : Array<Unknown<0>> should be Null<Map<Strin
    g, Dynamic>>
    Source/Main.hx:55: characters 48-50 : Array<Unknown<0>> should be Map<String, Dy
    namic>
    Source/Main.hx:55: characters 48-50 : For optional function argument 'options'
    

    Edit : c:\HaxeToolkit\colyseus-hx-master\src\io\colyseus\Client.hx

    			// M.E. @:generic    public function join<T>(roomName: String, ?options: Map<String, Dynamic>, ?cls: Class<T>): Room<T> {
            //this.room = this.client.join("state_handler", [], State);
    		this.room = this.client.join("state_handler", new Map<String,Dynamic>(), State);
    

    Now the lime build project.xml html5 followed by lime test html5 work with the externs by Serjek.

    That leaves me the task of creating a new Stencyl Extension that utilizes the
    this.room.state.players.onAdd = function(player, key)
    methodology rather than the
    this.room.listen("players/:id", function(change) {
    methodology.



  • Haxe Client and Server on v0.10

    Using the externs by Serjek I was able to create a very Simple Colyseus Server with a StateHandler Room that handles string data. The data is send to the clients and everything works as it supposed to.

    But only for HTML5 !!!

    I again have lots of problems to get it to work on anything else than HTML5.

    Here are the (non-stencyl) HaxeToolkit steps that I made:

    set HAXE_PATH=c:\HaxeToolkit\haxe
    set NEKOPATH=c:\HaxeToolkit\neko
    set PATH=%HAXE_PATH%;%NEKOPATH%;%PATH%
    haxelib setup c:\HaxeToolkit\haxe\lib
    haxelib --global update haxelib
    

    haxelib is up to date
    haxelib install openfl
    8.9.1
    haxelib run openfl setup
    7.5.0
    haxelib run lime setup
    Up to Date

    Get GIT from :
    https://git-scm.com/download/win

    git clone https://github.com/colyseus/colyseus-hx.git
    cd \HaxeToolkit\colyseus-hx\example\openfl\Source
    notepad Main.hx
    

    Edit: the ws connection so that it points to the docker server
    Also Edit the this.client.join line on number 55:

            //this.room = this.client.join("state_handler", [], State);
    		this.room = this.client.join("state_handler", new Map<String,Dynamic>(), State);
    

    Build the project:

    cd \HaxeToolkit\colyseus-hx\example\openfl
    haxelib install haxe-ws
    haxelib set hxcpp 3.2.94
    

    (installing)

    lime build project.xml html5
    lime test html5
    

    It works like a charm with the Serjek externs in the Colyseus Docker .

    But Android and Windows give again connection problems:

    cd \HaxeToolkit\colyseus-hx\example\openfl
    haxelib set hxcpp 4.0.8
    lime build windows
    
    lime test windows
    

    The connection is attempted but not made:

     - src/lime/utils/AssetCache.cpp  [haxe,release]
    Link: ApplicationMain.exe
       Creating library ApplicationMain.lib and object ApplicationMain.exp
    Main.hx:48: CLIENT CLOSE
    Connection.hx:66: WebSocket connection has been closed, stopping the thread!
    Main.hx:29: ERROR! timeout
    

    (I've tried the alterations on the haxe-ws library that I made for the 0.9 but they didn't solved it now)

    I am very curious if anyone got a different target to run with haxe other than HTML5 ??!??


  • administrator

    Hi @mdotedot, have you tried the haxe-ws fork here? https://github.com/colyseus/haxe-ws

    You can install it via:

    haxelib git haxe-ws https://github.com/colyseus/haxe-ws.git
    

    I've fixed the WS handshake on sys targets on this fork. Let me know if the CPP target works for you using it. Cheers!



  • @endel said in Stencyl (HaXe) Extension:

    https://github.com/colyseus/haxe-ws

    Endel you are a true Master!

    C:\HaxeToolkit\haxe\lib>haxelib git haxe-ws https://github.com/colyseus/haxe-ws.git
    Installing haxe-ws from https://github.com/colyseus/haxe-ws.git
    Library haxe-ws current version is now git
    
    C:\HaxeToolkit\haxe\lib>haxelib list
    actuate: [1.8.9]
    box2d: [1.2.3]
    haxe-ws: [git]
    haxelib: [3.3.0]
    hxcpp: 3.2.94 [4.0.8]
    layout: [1.2.1]
    lime-samples: [7.0.0]
    lime: 7.2.1 [7.5.0]
    openfl-samples: [8.7.0]
    openfl: 8.8.0 [8.9.1]
    

    Now to build / test it again:

    cd \HaxeToolkit\colyseus-hx\example\openfl
    lime build windows
    lime test windows
    

    Error:

    Can't find a secure source of random bytes. Reason: [file_open,\Device\KsecDD]
    

    Debugging leads me to this part

    trace("WebSocketGeneric.hx . initialize ... Before this.key 		");
            //this.key = Base64.encode(Crypto.getSecureRandomBytes(16)); // This generates the secure source of random bytes
    		this.key = Base64.encode(haxe.io.Bytes.ofString("ABCDEFGHIJKLMNOP"));
    trace("initialize key : "+this.key);
    

    After this ofString-"16-bytes" the error was gone and hand-shake was made!!!

    Windows and Android publication worked out of the box.

    I will test iOS/Mac later this week!


  • administrator

    Thanks for testing @mdotedot, that's nice. The problem is on getSecureRandomBytes method on Windows (unfortunately I don't have a Windows machine available for testing)

    This method is used to generate the Sec-WebSocket-Key header for handshake. The error comes from this line. I've borrowed the implementation from here.

    Would be great if you find a way to generate secure random bytes on Windows environment! Cheers!



  • The wikipedia linked in the sourcecode of Crypto.hx mentioned :
    "but reading the special file \Device\KsecDD does not work as in UNIX"

    So I tried another crypto number generator:

    haxelib install trandom
    

    Edit the project.xml to include this haxelib

    notepad \HaxeToolkit\haxe\lib\haxe-ws\git\src\haxe\net\Crypto.hx
    Change this:

    	 #if windows
    
    			var randomValue = trandom.TargetRandom.random();
    			var bytes = haxe.io.Bytes.alloc(length);
    		        bytes.setInt32(0, randomValue);
    			return bytes;
    
                        var input = sys.io.File.read("\\Device\\KsecDD");
    
                    #else
    
    

    This worked when I did a lime build & test for windows.

    When I tried to incorporate the trandom library into Stencyl it gave me problems. There is a define done that is going to add a build.xml file to the publication method and Stencyl does not handle that.

    I opted for another approach for the windows build from in Stencyl:

    		#if windows
    					return  haxe.io.Bytes.ofString(getRandomString(16));
    					// This will not work
                        var input = sys.io.File.read("\\Device\\KsecDD");
    
                    #else
    

    getRandomString:

    public static function getRandomString(length:Int, ?charactersToUse = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"):String
    	{
    		var str = "";
    		for (i in 0...length){
    			str += charactersToUse.charAt( Math.floor((Math.random() *  (Date.now().getTime() % (charactersToUse.length) ) )));
    		}
    		return str;
    	} //getRandomString
    

    It is not as strong as the crypto generator but at least it works.

    Confirmed v0.10 & Serjek Externs with Simple Stencyl Extension (Kind of like the NyanCat demo)

    • HTML5
    • Windows
    • Android
    • iOS Simulator
    • iOS on device
    • Macintosh OSX
    • (Oracle) Linux

  • administrator

    Awesome, thanks for sharing @mdotedot! I've just added your workaround for Windows into the haxe-ws fork: https://github.com/colyseus/haxe-ws/commit/eea2e57c53d8b2541475d66a9381e729f8f755d0



  • The past days I've worked on the Colyseus v0.10 framework and the Haxe externs from Serjek. As well as creating a docker that will run the server and will be supplied to the Stencyl Users so they can run the server on their own.

    Since Endel has provided a haxe-ws fork ( https://github.com/colyseus/haxe-ws ) the system is really stable.

    I've started with the core room-types:

    • Lobby
    • Raw
    • Turn
    • Lock

    Lobby is where all players are signed in automatically. The playernames are stored and can be used in other rooms.

    Raw Room is for games that don't need server logic. Think of games that are split screen and don't interact with eachother.

    Turn Room is for turnbased games.

    Lock system is where the server has a lock on coordinates/areas/objects and you request a lock to alter the data. Think of RPGs, puzzle games

    In the future I want to have a Box2D Server collision interaction, but that will propably take a long time to figure out.

    Currently a proof-of-concept has been made for the raw-room type. For the others I need to create more Stencyl blocks like request/release lock and turn (seat/next turn)

    Here is the current Stencyl palette layout:

    http://photoquesting.com/Colyseus/ColyseusStencylExtension_v0.42.png

    http://photoquesting.com/Colyseus/ColyseusStencylExtension_RoomInfo.png

    I've used client.getAvailableRooms

    for (room in rooms) {
    	if(room.metadata.ApplicationID == ""+colyseus.ApplicationID){
    		roomIDs.push(""+room.roomId);
    		roomNames.set(""+room.roomId,""+room.metadata.RoomName);
    	}
    }
    

    I wonder if there can be a filter for things like this so that I don't get all the rooms and filter the applicationID out 'manually'
    At least it works, but I don't know if it slows down when there are a lot of games running on the server?
    Then again, I don't think that many stencyl-developers will run a server with different type of games.


 

GitHub Documentation Patreon donate button

© 2019 Endel Dreyer