Stencyl (HaXe) Extension

This week I've made progress on a simple room where players can be joined and walk around.
I made it so that players would be moved simultaneous to see if the server could handle it.

It didn't quite run well with 2 or more browser sessions at the same time. Also a windows publication didn't fare well.

Yesterday I found out that the CPU on Windows and memory on HTML5 where draining. So today I made a simple version on my NyanCat project. I had it running on both html5 and windows and it had the same effect.

Then I stripped colyseus from the nyancat so that only the image was left with controlling keys and there was no problem with CPU.

With Colyseus:

And the non-colyseus version:

So Haxe client is consuming a lot of CPU !

At least it wasn't my client implementation since both have the same problem. I guess I need multiple computers from now on to do tests ....

Last days I've worked on refactoring code and CPU investigations.

Connection.hx modifications to avoid CPU consumption

 #if sys
		while(!this.isClosed){ //M.E. After 200 reconnections the CPU was gone to 92%
			this.ws.process();
				
			 Sys.sleep(0.005); // M.E. Avoid CPU consumption by introducing a really small sleep
        }
 #end
 public function close () {
		this.isClosed=true; //M.E. add for thread checking
        this.ws.close();
 }
		

Perhaps the this._enqueuedSend mechanism can be altered as well to check for the isClosed state?!?! But I'm not sure what the enqueued mechanism needs this...

Running with over more than 200 client initialization tests (and client.closing them) the CPU consumption kept relatively steady.
Now the Windows executable doesn't take much of the CPU.

Remains Javascript that consumes a lot of memory. But this couldn't easily be reproduced on the HaxeKit version, so it might be something related to other parts of HaXe and Stencyl.

I was able to switch rooms on Android, Windows and HTML5 though. I need to test the other targets with the latest build to make sure I didn't break anything for them.

Current state:

• Initialize Colyseus (tested 200x re-initializations)
• Create Room (Lobby, State_Handler) : Also done together with the 200x initializations
• Get room list (getAvailableRooms)
• Join Room
• Leave Room
• Player list
• Send Data
• Receive Data

TODO:

• Investigate Javascript memory consumption
• Publication to all Stencyl targets
• Ping / latency mechanism
• Broadcast investigations
• onAuth investigations
• Different Room Types : RawData,TurnBased,LockData,SimplePhysics,Chat

• javascript memory consumption was a Stencyl bug. It has to do with drawing stuff, which I obviously did in my test application. Filed a PR on Stencyl Forum for this.
• Publications:
  iPhone Simulator
  iPhone (Should be fine on iPad as well)
  Mac OSX
  Linux
  Windows
  Android (Tablet+Phone)
  HTML5

So all is fine to continue working on the RoomTypes.

Hello! do you use haxe only for client code or you have created externs for server as well?

Hello Serjek,

Currently I'm working on Stencyl Extension which uses (modified) HaXe Colyseus Client code.
The server is still running on TypeScript. A HaXe server would be awesome to have but I currently have my hands full with Client Code !

Kind regards from
M.E.

This weekend I made:

• PingTime
• PlayerID from SessionID block
• Player in Seat : This has to be changed to server-side, since it now apparently goes wrong sometimes (see pictures)

Made a milestone today: LockRoomType

This is one of the main goals of the Stencyl Extension.
I have wanted a lock-mechanism behavior for a long time.
The idea is that you have a grid-based game where the players 'lock' the part of the grid that will be changed.
When a lock is granted the player can alter the grid (move/push/pull/alter state).
Once data has been modified an unlock is send.

What I have is still flawed by leaving/killing or anyother type of disconnection.
When there is a lockstate it will not be unlocked.
That will be altered by a time-exceeded-system so that the server will unlock the set of grid positions
that the player/client had locked.
I probably need to use a 'previous' data mechanism as well so that data can be reversed when it only partially has been changed.

Made a prototype so that moves are done automatically.

Current Progress:

Hmm... will need to double check system because running for over an hour it created another block :(

At least the player positions (the colorblocks) are in sync!

This week I've worked on :

• Player in Seat - ServerSide
• TurnBased RoomType: NextTurn, isTurn
• MasterPlayerID for Client Side logic Room Type
• Lock Mechanism

The lock mechanism was flawed by not release locks and to not compare clientside grid with serverside grid.

CLIENT
For example when the client pushes a block from position 25 to 15 it will need to send 15=0,25=2,35=1000 to the server and request a lock on them:

grid[15]=0; 	// free space
grid[25]=2; 	// block
grid[35]=1000; 	// playernr

SERVER
Server receives the lockrequest (15=0,25=2,35=1000)
It checks his own grid if the positions match the client request lock. If the grids are out of sync it will send an error back.
The client will then request a complete grid from the server as it is apparently behind.

If the server grid position matches the requested lock positions a lock will be attempted: (pseudo-code)

attemptposition=new Array();
isOK=true;
for each lockposition{
	if(isOK && gridlock[lockposition] == 0){
		gridlock[lockposition]=time; 
		attemptposition.push(lockposition)
	}else{
		isOK=false;
		// there is a lock on one of the 
		for(each attemptpositon p) gridlock[p]=0;
	}
}

When the server was able to lock all gridpositions it will send a success to the client

CLIENT
The client will do a grid request if the return state was an error.
The client will send the changed data to the server when it was success, otherwise it will not do the move as there was/is a lock.

SERVER
When server receives the griddata it will set the new grid values and afterwards set the lockpositions to 0-time.

After each request the server checks the gridlock table if the currenttime - locktime > 2 seconds .. when it finds one it will reset the lock.

CLIENT
Sends unlock to server.

A room with four player-bots ran for over 4 hours and still they were in sync.

Next I will need to figure out how to do classes with overloading / inheritence in TypeScript.
I want all the room types to share code that are common in all rooms.

For instance I need PlayerInSeat, PingTime, getPlayersInRoom etc.. code in all typescript rooms.

Subject: Colyseus HaXe Externs & Colyseus version 0.10

Took a break from the Colyseus Extension, but I'm back.
I was asked to create a MicroPhone HaXe extension for iOS and Android so that did take some time.

Also, I wanted to wait a bit till the version 0.10 was done on both of the client and server side.

Thanks to Serjek and Endel for their contributions, I was able to get a HaXe combination running with the Nyan Cat example.

Server Side: Oracle Linux

• Download : https://github.com/serjek/colyseus-hxjs
• Download : https://github.com/serjek/colyseus-hxjs-examples
• extract in /root/serjek directory
• Install npm : curl -sL https://rpm.nodesource.com/setup_6.x | sudo -E bash -
• yum install -y nodejs`
• npm -version
  6.5.0-next.0
  HaXe
• Download install-haxe.hs : https://gist.github.com/jgranick/8cc40e2e0f277146725f
• sh install-haxe.sh
• export LD_LIBRARY_PATH=/root/haxe/neko
• haxelib setup
  /usr/lib/haxe/lib
• cd /root/serjek/colyseus-hxjs-examples-master/
• npm i lix -g
• lix download
• Upgraded haxe to 4.0.0 preview 4 from : https://haxe.org/download/file/4.0.0-preview.4/haxe-4.0.0-preview.4-linux64.tar.gz/
• Updated the install-haxe.sh to reflect that .tar.gz
• ran install-haxe.sh again
• haxe -version
  4.0.0-preview.4+1e3e5e0
• lix use haxe 4.0.0-rc.2
• Make sure that you are in the serjek/colyseus-hxjs-examples-master directory
• haxe server.hxml
• cd bin/server
• npm i yarn -g
• yarn
• node index.js
  src/MainServer.hx:39: -- listening on 0.0.0.0:2567... --

New Session

• export LD_LIBRARY_PATH=/root/haxe/neko
• /usr/local/bin/haxe
• lix use haxe 4.0.0-rc.2
• haxe -version
  4.0.0-preview.4+1e3e5e0
• cd /root/serjek/colyseus-hxjs-examples-master
• haxe client.hxml
• cd bin/client
• yarn
• node index.js

Test will run.

Now for the HaXe Client on Windows: https://github.com/colyseus/colyseus-hx

• https://github.com/HaxeFoundation/haxe/releases/download/3.4.4/haxe-3.4.4-win64.exe
• set HAXE_PATH=c:\HaxeToolkit\haxe
• set NEKOPATH=c:\HaxeToolkit\neko
• set PATH=%HAXE_PATH%;%NEKOPATH%;%PATH%
• haxelib setup c:\HaxeToolkit\haxe\lib
• haxelib --global update haxelib
  Current version is 3.3.0
• haxelib install openfl
  8.9.0
• haxelib run openfl setup
  7.3.0
• haxelib run lime setup
  Up to Date
• lime create HelloWorld
• cd HelloWorld
• lime test html5
• extract the colyseus hx master to \haxetoolkit\colyseus-hx-master
• cd \HaxeToolkit\colyseus-hx-master\example\openfl\Source
• notepad Main.hx
  Edit: the ws connection so that it points to the Linux Server mentioned above.
• cd \HaxeToolkit\colyseus-hx-master\example\openfl
• haxelib install haxe-ws
• haxelib set hxcpp 3.2.94
  (installing)
• lime build project.xml html5
  Error:

Source/Main.hx:55: characters 48-50 : Array<Unknown<0>> should be Null<Map<Strin
g, Dynamic>>
Source/Main.hx:55: characters 48-50 : Array<Unknown<0>> should be Map<String, Dy
namic>
Source/Main.hx:55: characters 48-50 : For optional function argument 'options'

Edit : c:\HaxeToolkit\colyseus-hx-master\src\io\colyseus\Client.hx

			// M.E. @:generic    public function join<T>(roomName: String, ?options: Map<String, Dynamic>, ?cls: Class<T>): Room<T> {
        //this.room = this.client.join("state_handler", [], State);
		this.room = this.client.join("state_handler", new Map<String,Dynamic>(), State);

Now the lime build project.xml html5 followed by lime test html5 work with the externs by Serjek.

That leaves me the task of creating a new Stencyl Extension that utilizes the
this.room.state.players.onAdd = function(player, key)
methodology rather than the
this.room.listen("players/:id", function(change) {
methodology.

Haxe Client and Server on v0.10

Using the externs by Serjek I was able to create a very Simple Colyseus Server with a StateHandler Room that handles string data. The data is send to the clients and everything works as it supposed to.

But only for HTML5 !!!

I again have lots of problems to get it to work on anything else than HTML5.

Here are the (non-stencyl) HaxeToolkit steps that I made:

set HAXE_PATH=c:\HaxeToolkit\haxe
set NEKOPATH=c:\HaxeToolkit\neko
set PATH=%HAXE_PATH%;%NEKOPATH%;%PATH%
haxelib setup c:\HaxeToolkit\haxe\lib
haxelib --global update haxelib

haxelib is up to date
haxelib install openfl
8.9.1
haxelib run openfl setup
7.5.0
haxelib run lime setup
Up to Date

Get GIT from :
https://git-scm.com/download/win

git clone https://github.com/colyseus/colyseus-hx.git
cd \HaxeToolkit\colyseus-hx\example\openfl\Source
notepad Main.hx

Edit: the ws connection so that it points to the docker server
Also Edit the this.client.join line on number 55:

        //this.room = this.client.join("state_handler", [], State);
		this.room = this.client.join("state_handler", new Map<String,Dynamic>(), State);

Build the project:

cd \HaxeToolkit\colyseus-hx\example\openfl
haxelib install haxe-ws
haxelib set hxcpp 3.2.94

(installing)

lime build project.xml html5
lime test html5

It works like a charm with the Serjek externs in the Colyseus Docker .

But Android and Windows give again connection problems:

cd \HaxeToolkit\colyseus-hx\example\openfl
haxelib set hxcpp 4.0.8
lime build windows

lime test windows

The connection is attempted but not made:

 - src/lime/utils/AssetCache.cpp  [haxe,release]
Link: ApplicationMain.exe
   Creating library ApplicationMain.lib and object ApplicationMain.exp
Main.hx:48: CLIENT CLOSE
Connection.hx:66: WebSocket connection has been closed, stopping the thread!
Main.hx:29: ERROR! timeout

(I've tried the alterations on the haxe-ws library that I made for the 0.9 but they didn't solved it now)

I am very curious if anyone got a different target to run with haxe other than HTML5 ??!??

Hi @mdotedot, have you tried the haxe-ws fork here? https://github.com/colyseus/haxe-ws

You can install it via:

haxelib git haxe-ws https://github.com/colyseus/haxe-ws.git

I've fixed the WS handshake on sys targets on this fork. Let me know if the CPP target works for you using it. Cheers!

@endel said in Stencyl (HaXe) Extension:

https://github.com/colyseus/haxe-ws

Endel you are a true Master!

C:\HaxeToolkit\haxe\lib>haxelib git haxe-ws https://github.com/colyseus/haxe-ws.git
Installing haxe-ws from https://github.com/colyseus/haxe-ws.git
Library haxe-ws current version is now git

C:\HaxeToolkit\haxe\lib>haxelib list
actuate: [1.8.9]
box2d: [1.2.3]
haxe-ws: [git]
haxelib: [3.3.0]
hxcpp: 3.2.94 [4.0.8]
layout: [1.2.1]
lime-samples: [7.0.0]
lime: 7.2.1 [7.5.0]
openfl-samples: [8.7.0]
openfl: 8.8.0 [8.9.1]

Now to build / test it again:

cd \HaxeToolkit\colyseus-hx\example\openfl
lime build windows
lime test windows

Error:

Can't find a secure source of random bytes. Reason: [file_open,\Device\KsecDD]

Debugging leads me to this part

trace("WebSocketGeneric.hx . initialize ... Before this.key 		");
        //this.key = Base64.encode(Crypto.getSecureRandomBytes(16)); // This generates the secure source of random bytes
		this.key = Base64.encode(haxe.io.Bytes.ofString("ABCDEFGHIJKLMNOP"));
trace("initialize key : "+this.key);

After this ofString-"16-bytes" the error was gone and hand-shake was made!!!

Windows and Android publication worked out of the box.

I will test iOS/Mac later this week!

Thanks for testing @mdotedot, that's nice. The problem is on getSecureRandomBytes method on Windows (unfortunately I don't have a Windows machine available for testing)

This method is used to generate the Sec-WebSocket-Key header for handshake. The error comes from this line. I've borrowed the implementation from here.

Would be great if you find a way to generate secure random bytes on Windows environment! Cheers!

The wikipedia linked in the sourcecode of Crypto.hx mentioned :
"but reading the special file \Device\KsecDD does not work as in UNIX"

So I tried another crypto number generator:

haxelib install trandom

Edit the project.xml to include this haxelib

notepad \HaxeToolkit\haxe\lib\haxe-ws\git\src\haxe\net\Crypto.hx
Change this:

	 #if windows

			var randomValue = trandom.TargetRandom.random();
			var bytes = haxe.io.Bytes.alloc(length);
		        bytes.setInt32(0, randomValue);
			return bytes;

                    var input = sys.io.File.read("\\Device\\KsecDD");

                #else

This worked when I did a lime build & test for windows.

When I tried to incorporate the trandom library into Stencyl it gave me problems. There is a define done that is going to add a build.xml file to the publication method and Stencyl does not handle that.

I opted for another approach for the windows build from in Stencyl:

		#if windows
					return  haxe.io.Bytes.ofString(getRandomString(16));
					// This will not work
                    var input = sys.io.File.read("\\Device\\KsecDD");

                #else

getRandomString:

public static function getRandomString(length:Int, ?charactersToUse = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"):String
	{
		var str = "";
		for (i in 0...length){
			str += charactersToUse.charAt( Math.floor((Math.random() *  (Date.now().getTime() % (charactersToUse.length) ) )));
		}
		return str;
	} //getRandomString

It is not as strong as the crypto generator but at least it works.

Confirmed v0.10 & Serjek Externs with Simple Stencyl Extension (Kind of like the NyanCat demo)

• HTML5
• Windows
• Android
• iOS Simulator
• iOS on device
• Macintosh OSX
• (Oracle) Linux

Awesome, thanks for sharing @mdotedot! I've just added your workaround for Windows into the haxe-ws fork: https://github.com/colyseus/haxe-ws/commit/eea2e57c53d8b2541475d66a9381e729f8f755d0

The past days I've worked on the Colyseus v0.10 framework and the Haxe externs from Serjek. As well as creating a docker that will run the server and will be supplied to the Stencyl Users so they can run the server on their own.

Since Endel has provided a haxe-ws fork ( https://github.com/colyseus/haxe-ws ) the system is really stable.

I've started with the core room-types:

• Lobby
• Raw
• Turn
• Lock

Lobby is where all players are signed in automatically. The playernames are stored and can be used in other rooms.

Raw Room is for games that don't need server logic. Think of games that are split screen and don't interact with eachother.

Turn Room is for turnbased games.

Lock system is where the server has a lock on coordinates/areas/objects and you request a lock to alter the data. Think of RPGs, puzzle games

In the future I want to have a Box2D Server collision interaction, but that will propably take a long time to figure out.

Currently a proof-of-concept has been made for the raw-room type. For the others I need to create more Stencyl blocks like request/release lock and turn (seat/next turn)

Here is the current Stencyl palette layout:

I've used client.getAvailableRooms

for (room in rooms) {
	if(room.metadata.ApplicationID == ""+colyseus.ApplicationID){
		roomIDs.push(""+room.roomId);
		roomNames.set(""+room.roomId,""+room.metadata.RoomName);
	}
}

I wonder if there can be a filter for things like this so that I don't get all the rooms and filter the applicationID out 'manually'
At least it works, but I don't know if it slows down when there are a lot of games running on the server?
Then again, I don't think that many stencyl-developers will run a server with different type of games.

With the core functionality done, the next room was much faster to create.

RoomType: Turn Based System

Proof Of Concept (sorry for the wall of text which is used to debug stuff)

This leads to these new blocks in Stencyl:

When a player leaves the system will pick the next seat as the active seat. If there is no seat after the leaving player the first available seat is selected.

Also, the seat that becomes availabe is added to a stack for next players to join to. (Of course when the Stencyl Developer decides to allow join/rejoining of active game!)

The Lock System is still a good challenge. After two code-refactors I have put it on hold for now.

Last week I've worked on an actual TurnBased system game (TicTacToe)

[running html5 and windows publication next to eachother]

(published for HTML5, Windows and Android; should work on the others)

During development I started to rethink my approach of a single extension versus multiple extensions.
I believe that a Stencyl user wants to create a specific type of game and install / enable the extension that is used for that type of game.
This make the palette a lot cleaner. (Unfortunately Stencyl Extension Developers don't have the means to create sub-palettes or tabs)

TODO for TurnBased System :

• set timeout value; when a player needs to react before certain time (auto-logout)

TODO for general :

• make roomtype specific extensions (raw, turn, physics, lock/logic) that have still the general purpose blocks (init, join, create, leave room, etc..)

TODO for games:

• Make another turnbased game (Poker?)

Also, I need to figure out the Lock/Logic room type eventually...

Working towards a beta release of the Colyseus Stencyl Extension.

I had already Lock, TurnBased and Raw room-types.

Most of the progress has been made with the Client-As-A-Server concept.
I know that the best way to handle multiplayer is on the server-side.
But many of the Stencyl users are not comfortable writing the server-side logic.

Therefore I attempted for a Client-Server-Relay kind of thing.
The collision and logic is handled on the client side with one of the clients acting as the server.
On that client the real physics objects are hidden and only the data that is send to all players is displayed.

These are the things that are currently made with the Client-As-A-Server approach:

Left Window is a Windows executable
Right Window is a web-page

I hope to do some more beta-testing with other Stencyl users to gain more information about this approach.

Since I want to join the Ludum Dare Jam 47 (2nd October) and make an on-line game we need to make resources public available.
Therefore I have made the Stencyl Extension available as a public beta : Stencyl Colyseus Extension Page

Last days I've worked on the documentation

And I worked on implementing a turn based card game.

Sending and receiving data is a breeze with Colyseus.

It all depends on the game-logic and how to show the state of the game.

I hope that the docker container that I provide to the Stencyl users will make running of the server easier for them.

As mentioned before I have made the extension based on the work of serjek (haxe externs) on top of the Colyseus engine.

The server has only three logic parts : Turn based and Locking mechanism and a check on active seat.
All other logic has to run on the client.

Of course most good server implementations need more logic on the server,
but I still am not certain how to provide the stencyl users with a relative easy way to make server-side code.

The best 'feel' you get is when you implement the Lock Room Type for your game. The room-data is kind of like a database server
that allows only one player to modify the data.

But, to be honest, most Stencyl users want to create physics based games. I provide examples how to run a client-as-a-server approach.
That seems to work since all clients see the same 'server'-state. But it is never as fast as it could be with server-side-physics.

I've attempted to create physics logic on the server based on the Box2D library.
The problems arise when doing client side prediction and server lag compensation. I wasn't able to get that to work. The data on the
clients 'jumped' all over the place. It works when you have minimal player interaction on the physics objects, but then the 'client-as-a-server' approach
is way easier to implement.

While preparing for the LD47 I revisted an old (LD38) project and converted it into a Client-As-A-Server game.

The left shows player control using the arrow keys and the right session you control the rock with the mouse.

The Extension Demo Page where you can play yourself is:
StencylColsyeusDemoPage