r3dc0d3-blog

[r3dc0d3-blog]

Devember Day 7, 8, 9 DevLog

So, I disappeared for the last 3 days. I was sick. Luckly I already invested 15 hrs on the project so technically my 1hr/day is still valid and I still have excess time. Anyway...

A problem kicked in.

For some reasons my C PInvoke functions stopped from working the right way. The console would execute the commands but no output was on the screen. I was baffled. My original code for getting the console handle was:

IntPtr handle = Process.GetCurrentProcess().MainWindowHandle;

Well, that line is now returning 0. Time to use even more C functions. There goes CConsole.GetConsoleWindow();, a PInvoke of the respective C functions, that returns a proper handle, but still does nothing, my write functions do not write anything on that handle.

But it was working few days ago, wtf is going on?

Well, I'm still trying to solve this.

See you on the next DevLog

[r3dc0d3-blog]

Devember Day 6 DevLog

It's color time!

How C console methods manage colors

When you write on the buffer using WriteConsoleOutput you pass a 2D array of CHAR_INFO, a struct that contains an union of an ASCII and Unicode char and a UInt16 containing the attributes of the character.

the UInt16 layout is very simple, the last 8 bits are the foreground and background color in IRGB format, where I is if the color is intense. the remaining 12 bytes are for Chinese, Japanese and Korean charsets.

BG FG C J K IRGB IRGB 0000 0000 0000 0000 0000

An that's all for today.

Yeah, spent a couple hours figuring out how it worked and that's it. Tomorrow I will hopefully draw something on the screen, maybe with the help of a couple of extension methods to convert things easily between C# and C types.

[r3dc0d3-blog]

Devember Day 5 DevLog

The interop hell

So, yesterday I was writing out some interop code to access the console buffer using the winapi functions. I wrote a couple of interop types (SMALL_RECT and COORD) and the proper PInvoke code for WriteConsoleOutputW.

//interop type [StructLayout(LayoutKind.Sequential)] public struct SMALL_RECT { public UInt16 Left; public UInt16 Top; public UInt16 Right; public UInt16 Bottom; } //interop type [StructLayout(LayoutKind.Sequential)] public struct COORD { public UInt16 X; public UInt16 Y; } [DllImport("Kernel32.dll", EntryPoint = "WriteConsoleOutputW")] private static extern void WriteBufferOutput(IntPtr consoleOutput, char[,] data, COORD bufferSize, COORD startCoord, out SMALL_RECT WriteRegion);

but I didn't know if it was working. And then something was running on my mind..

I might as well create a wrapper for the console commands in C or C++.

And that's the plan! Time to add a new project in the solution. And it was at that point that I looked at the clock.

It's 1:38AM! I'm already at day 5.

But at least, I know what I'm going to do. It's time to learn about interop, the C commands for console handling and all it's keeping me away from a performant C# console output.

Getting hands dirty with Interop services

At this point I'm basically working on a wrapper of the C console functions to use them on C# due to the C# api not having a way to directly write to the buffer.

Removed the interop code from the Engine project, it was also time to add a separate, project for testing out theese 2 libraries.

My plan now

I'm now going to get the wrapper working first, meanwhile learning how interop works. When the console wrapper is done I can return to finish the engine renderer itself.

I can also work to a windows audio wrapper to work on the audio side if the C# functions for playing audio aren't decent enough. Or I can use some extern library to do the dirty work for me.

Looking back at the whole situation, I'm basically writing a middleware for my engine and already planning for a possible second middleware. Damn, this is really getting out of control.

Getting a bit of luck

Might not be the best solution, noether the most tested solution. But hey, in my search for a way to avoid PInvoke hell, I found a really useful website. PInvoke.net, containing basically al the code I needed for all the console functions in one simple spot. With a warning saying that they aren't really tested. Well, time to test them.

Pasting 760+ codelines has a bad feeling to be honest...

After the copy, IntelliSense is marking me only 3 errors. Nice!

One error was about a struct having unsafe code and was easy to fix with a flag to the compiler.

Another one was a typo on a matrix declaration, again, easy to fix.

The last one was about passing an uint as int without a typecast. Another easy fix.

Then I hit that Compile button and holded my breath.

And as Compiled Succesfully showed up on my bottom left of the screen my face changed from a doubtful expression to a wide smile...

Well, time to test it, but first I have to clean up the code I pasted.

It's all about the RGB, baby!

ConsoleColor on the C# api only has 16 colors. The C console functions have full RGB capabilities. This is a game changer addition.

Errata corrige here. It still has 16 colors, but you can choose the rgb of each of them. I'm not going to bother with all of that for now.

The great cleanup

Those 760 lines were a class and a lot of structs. I ended up moving by hand all of the structs to their respective files, I reworked a couple of them (the one that had unsafe code now is safe.) and kept on refactoring until all the project started to get manageable.

C console has mouse support too!

Yeah, looks like C console functions have methos to get the mouse position and event a mouse event. Perfect. Can't wait to even have mouse working on a console app.

And then it's testing time

My advanced console class is without any documentation right now. I should fall back to the microsoft kernel32.dll documentation for C. I'm just going to test all the methods and write their documentation as I need to use them. It's gonna take time, a lot of it, but gradually I should have everything documented and working.

This was my first test of the C console methods. An hello world, written from a C function, and it worked like a charm!

A long documentation night

It's 10:45PM at the time of writing this, and from the looks of it, it's going to be a long documentation night.

This sums up Day 5 of my Devember challenge, an interesting day to be honest, where I got more in-depth knowledge of how PInvokes work and discovered new ways to expand this engine capabilites.

I write the post during the day as I find things to report in. In this way when I end the coding session, I just have to write a couple of things and the whole post is ready, this way is easier to not forget anything.

[r3dc0d3-blog]

Devember Day 4 DevLog

It's time to Render things!

Or so I tought... turns out things were a bit more challenging than I tought...

Breaking down the rendering process

Rendering an image in the console it's easy, rendering multiple sprites accounting for transparency and color it's a bit more complicated.

The whole buffer is just a 2D array of AsciiPixels. A fancy name for a char paired with a ConsoleColor value in a struct. I could have used a Tuple to be honest, but didn't think about it and it looks cleaner this way.

To calculate the proper final buffer to be rendered I need to overlay the varius sprites, taking account for their sorting order. My Transform class takes care of this, giving me the X, Y and Z position of an IRenderable object. I first sort the List of IRenderable in descending order using a Comparer with an lambda expression, to avoid creating a whole new List object every time, then I start writing to the buffer the corresponding chars of the sprites skipping the ones that are blank.

After this whole operation I have a 2D array of the final frame ready to be rendered on the buffer. (All the previus operations are made on a temporary buffer, before being written on the actual console buffer).

Now I need to write them to the buffer. Doing multiple write operations are the best way to cripple performance. The best way to go seems to be doing multiple color passes (as you can only write a single color each write operation) and then we are set.

Separating the color passes

ConsoleColor enum has 16 different value. 7 colors with their respective dark variant plus black and white.

At this point I just realized I didn't take in account that you can combine 2 colors each time, there is a background color and a foreground color. this brings the number of combinations up to 256. My original idea was to just allocate a buffer for each color, but if I want to have access to the background color (that until now I just forgot about it), I would need 256 buffers. That's a lot of write operations. I can't actually do this that way. A more pratical way is to create a list of buffers of each color contained in the frame (allocating it only if I need that) and hope I will never use too much colors at the same time.

In the end I decided to leave alone the background color for now and just move on.

I create a dictionary of ConsoleColor, char[,] to easily create just the buffers I need.

And it was at this moment that Jackson knew... He fucked up.

Writing to a specific position in the console buffer? Isn't gonna happen. There are no methods to write in a specific position in the buffer without moving the cursor.

Now, the documentations says:

If the specified cursor position is outside the area that is currently visible in the console window, the window origin changes automatically to make the cursor visible.

And this will just make everything choppy. Oh well it's time to start using PInvokes and access the proper console buffer.

And this was my first time calling C code from C# (and my first time using interop services at all...)

And it's not going well... At all...

Calling C functions from C#

It looks easy at first, but has some hidden painful drawbacks.

On the surface you just use a [DllImport()] attribute, specifing the dll you want to acces, the entry point and a couple of optional values.

Then you declare an extern static function that matches the one you want to call. And this can be a problem.

I'm trying to call the WriteConsoleOutputW C function, and it needs an handle. but there is no class for an handle in C#, and there is no clear table of how you should give to the function that handle.

It's getting late here and I'm getting tired of googling up this interop documentation. I'll see more tomorrow.

Until then... Happy coding!

[r3dc0d3-blog]

Devember Day 3.5 Devlog

I came back home earlier than I tought, so I started writing more code. And also started encountering the first problems...

The Renderer Idea

I wanted to create a rendering engine for the system console, but I also wanted to make it function just like modern game engines...

To make the renderer work without having tearing problems due to slow update speed (the console write operations aren't the fastest thing in the world...) I need to render the frame before and then move the buffer area to reveal it. Basically a double buffered rendering. Something that usually is standard in modern day GPU rendering.

Naive me tought it would have been easy to implement such a thing in a console.

So I started writing the Renderer interface.

The IRenderer interface looks like this:

public interface IRenderer : ISystem { public int TargetFPS { get; set; } public void DrawBuffer(int bufferIndex); public void SwapBuffer(int topBuffer); }

And up to this it was easy. Implementing it I noticed a couple of problems... My AsciiBuffer struct was not ready for the buffer copy.

Naive me strikes again...

I tought moving the buffer was the same thing as scrolling the window... Oh good lord how much I was wrong...

In fact there is no method for scrolling the console buffer in the window. Thinking about it is a good thing. Having to scroll the window up and down a dozen times per second (because I don't think I can have an higher frame rate... that 12-15FPS) would surely not be a pretty solution.

Instead System library gives us an useful MoveBufferArea() method...

And it's basically a copy operation. I can't be more happy! A copy operation should be fast and enables me to write to whatever buffer position for whatever reason and then just copy over to the visible buffer.

As I said, I was happy, until I looked at the required parameters:

void MoveBufferArea(int sourceLeft, int sourceTop, int sourceWidth, int sourceHeight, int targetLeft, int targetTop);

Then my first "Oh shit!" moment happened. Until now, thinking that I could simply scroll the window buffer I only saved the row where the buffer itself was starting.

"Well, back to the drawing board!"

And it's time to re-engineer the AsciiBuffer struct.

It looked like this before the rewrite:

public struct AsciiBuffer { public AsciiPixel[,] Data { get; set; } public int StartPosition; public AsciiBuffer(int X, int Y, int startPosition) { Data = new AsciiPixel[X, Y]; StartPosition = startPosition; } }

I need to have a quick way to get the 4 dimensions of a buffer placed somewhere in the buffer area so, instead of saving 4 different variables I decided to make a struct.

public struct BufferPosition { public int LeftStart { get; set; } public int TopStart { get; set; } public int Width { get; set; } public int Height { get; set; } }

This brought me to rewrite the whole constructor of the AsciiBuffer struct.

And it was at this point that I started using Left, Top, Width and Height as the standard way of referencing positions in the console.

X and Y were leading me to errors often, forgetting what was the top and what was the left. Much better, and only a bit more long than the usual.

All fine until I discovered the existance of another useful method. Console.SetWindowPosition(); This useful method just moves the window to show a different part of the buffer. Without moving the window.

Okay I suck at explaining things...

Basically the window is not moving at all... We are theorically moving the buffer relatively to the window, that is equal to moving the window over the buffer, that means that the window is not moving and. Okay, let's stop. I need to get some sleep.

I'll have to do a couple of benchmarks of the two methods. However both can be used, if I only have to update a small part of the screen could be more useful to just copy over a piece of the buffer rather than moving the whole window position on the buffer.

Anyway, for today I think it's enough...

Let's wrap it up.

So, today has been the first day of writing proper code, and it was fun. I'm getting a throwback to the late 80's way of creating games, with a couple of differences, like having a more high-level language to program games and having no constraints of ram.

Extra things done today not covered here

I created an IRenderable interface. The Renderer holds a list of them and cycles trough the list to render each one at the right position. To save the position of each IRenderable object I created a Transform struct, holding X, Y and Z coordinates of a sprite and also containing a property called Rotation.

Rotation is just an enum containing the rotation an object should have when rendered. It can have one of 13 values.

normal, clockwise45, clockwise90, clockwise135, clockwise180, clockwise225, clockwise270, counterclockwise45, counterclockwise90, counterclockwise135, counterclockwise180, counterclockwise225, counterclockwise270

How to rotate these 2D char arrays is still to be discovered. That's what I love of this challenge.

Anyway, the Devember challenge is really getting more and more interesting as I discovered that half the fun is in writing theese blog posts and trying to keep them with a bit of self-irony and humor. It's also a good way to practice my writing skills in english (as my native language is Italian) and having fun with Markdown formatting.

[r3dc0d3-blog]

Devember Day 3

Work is getting done. I now have a repository at https://github.com/SamueleLorefice/REDASCII-Engine that will be updated constantly. I could not get forward on the planning so I started writing some interfaces and classes. It ‘s been a long day and had less than 1 1/2 hours today to work on the engine.

I can probably work more later this night. For now I don ‘t have any other infos.

Until next time,

Happy coding!

[r3dc0d3-blog]

An update about day 2

So, the day 2 didn't go as planned. Had literally zero time to think about the engine. However on the first day I dedicated more than 4hr over the project so, I'm still good (the contract says 1hr a day. I already have 4 in 2 days. So, I'm fine.)

We will see some updates hopefully tomorrow.

[r3dc0d3-blog]

Devember Day 1 DevLog

So far so good, the first day of the Devember challenge is ending. This first day has been mostly around planning rather than proper code writing. I started laying down the basics of how the game engine should work. I started working on a mind map trying to explain exactly what every component is supposed to do.

Rendering on the console has some challenges, for example each “Pixel” is basically a character, so along with the color I also need to keep reference of the character it should display. Writing on the console in C# is not the most efficient way to do it (neither was years ago, smooth scrolling without GPU buffer access was literally impossible back in the day, and still is today...). So to keep framerate up I have 2 constraints: reduce the number of writes on the screen, and keep down the amount of colors at the same time.

As the Console.WriteLine() method can’t write multiple colors at once, I can only draw 1 color per pass. Limiting myself to 2/4 colors looks reasonable but also calculating and redrawing only the changed pixels from the previus frame will certainly help.

I decided to create this engine modularly with extensibility in mind. 

I defined 4 main systems that should run indipendently from each other. Those systems are:

Graphic System

Audio System

Input System

Messaging System

I think their names are pretty much self explanatory. The Messaging System handles communications between the systems and the actual game code.

I made a map of how this is going to work and I’ll update it while adding new interfaces and classes to the project.

I’ll add more details tomorrow. For now I still haven’t got a proper idea to handle all of this, it’s all experimental and subject to changes.

I’ll add more details tomorrow, toghether with a github link, until then,

happy coding!

[r3dc0d3-blog]

My Devember contract

I, Samuele Lorefice aka R3DC0D3, will participate to the next Devember. My Devember will be about creating a game engine for system console. I promise I will program for my Devember for at least an hour, every day of the next December. I will also write a daily public devlog and will make the produced code publicly available on the internet. No matter what, I will keep my promise.

More details down here:

The engine itself will focus on 2D graphics.

It should handle just position and rotation of the ASCII sprites.

ASCII Art only.

Focus on the whole screen composition with things like transparency, borders and clean rendering.

Performance is a bonus point but not required. For what I know from previus experience Console.WriteLine is not really optimized if you are rewriting the whole screen, a better way would be direct access to the console buffer.

The whole Project will be developed over .Net Core, using C# as usual.

Bonus point for not using C++ code or C functions calls for optimization.

There should be different subsystems running at the same time (graphic, sound, input) just like any other game engine.

Bonus point for not using external libraries.

The project will be publicy aviable on my GitHub profile.

Over the course of making this engine I will also program varius games to test the engine itself and also help shaping it in the right way.

Some of the games I want to create are: Pong, Tic Tac Toe, Space Invaders, Tetris.

Each day at the end of the programming session (I might largely exceed the 1 hours a day, as being unemployed has some advantages) I will create a tumblr post describing all the changes and considerations of the day.