Zeta Engineering HCS_C Status
8/29/08: Found a few more minor issues, I was able to fix some of them with this firmware: hcs_xpress_080829.txt, but unfortunately, something still isn't right with the power-up auto loading of the events file and with logfile retrieval. I'll have to put HCS_C aside for a few weeks but then I'll get these problems addressed.
8/27/08: OK, the 1.09b bits are here HCS_Software. This look very solid now, a week of testing shows that the fixes look good. There was only one offending variable that caused the re-entrancy problem--I was afraid it was going to be a lot worse. But along the way, with a bunch more testing, I found a few other problems, and the asterisk over-range output is a real pain when displaying hex (generally, you dont care about overrange in hex mode, you just want to display a fixed set of nibbles regardless of the upper nibbles), so I disabled that in that mode. One huge surprise was the discovery that the new WAIT capability in 1.08 broke the ability to save and restore power up events files! I have typically used timers, but this new WAIT capability is so powerful that now I use it all the time and discovered the problem when I set a new powerup events file. It's fixed now. If you see a problem, don't assume that I know about it, send me a note! For 1.09b, you need the new host, compiler, and FW. I'll get source up before too long. There's a slightly updated HCS_C_Compiler Users Guide too.
One thing I did discover--it is really easy to crash the HCS_C using bad formats in special escape sequences. Since the compiler will output anything as a string, it does not check for bad escape sequence strings--I need to add that in a future version of the compiler. But for now, on 1.09b, you must follow the syntax exactly within an escape sequence, no extra spaces, no variations. Once the syntax is correct, HCS_C has been exhibiting the reliability you all expect--but watch out for this right now.
I will not be working on HCS_C for a few weeks, my attention will be elsewhere--but when I finish that, I will get this network card out. Take a look online at the hcs-morrison site--it's still very crude, but all of the sensors are connected now. The counts are wrong, though, and several other things aren't working--but it gives you an idea of the power of the new string escape syntax.
8/24/2008: Oh my. I found two problems, first, the log function fails (causes lockup) when the log file size limit is hit. The second was a very nasty one that has always been there, rare, but significantly aggravated when you add multiple remote cards and ethernet. The streaming string processing routines are non-re-entrant, which I thought was OK because they are never called except by interrupt service routines in response to a transmit character ready (to the console, HCSNet, or other serial devices). Bad assumption--the output to the video routine is instantaneous, it just calls the stream string routine continuously and outputs straight to video RAM. It is getting interrupted by the console output or HCSNet output, and does a re-entrant call to a non-re-entrant routine. Ecccch. In all the last few years of continuous operation, I never got hit with this!
I'm working on a fix. You really want to do this bug fix to 1.10, all prior versions will potentially fail. I will update anybody for the cost of shipping, usually around $15 or so.
8/21/2008: After a new test, I'm seeing a nasty little hang in the console routine, you should wait on 1.09 until I get this figured out...
8/19/2008: oops, forgot to create windows for all the new supported devices in host, you'll want to pick up host_080818a.exe
8/18/2008: 1.09 is ready! The bits are here: HCS_Software
I have run quite a few tests and am thinking it looks pretty good, so it should get posted here tonight or maybe tomorrow night. Those of you that have systems here for updating will see the updates applied and shipped out in the next few days--some of you have been waiting so long! Thank you for your patience, I think you will be rewarded for the wait.
There's now so many display modes and formats and new things to check that you should be ready for some turmoil. The HCSNet interface in particular required a lot of fixes, so hopefully it's more usable now. Note that the bit assignments should now correctly match the connection users interface.
You may be wondering why I have so many display modes, especially for conditional strings, when I could just have the events program do the formatting before sending the string. The number one reason is to permit fast complex web pages--conditional strings are especially handy here because there's no "computer" when displaying a web page, computations can only be done with callbacks (requiring time consuming internet packet exchanges) or if there are builtins such as provided by the various HTML supersets. By providing these functions as part of the construction of the outgoing webpage, I ensure very fast page display and easy, yet fancy, page generation with dynamic content being sent out on a static page.
8/16/2008: Now the conditional strings work correctly with the new format. It will work correctly in the new <object><format> string escape sequence syntax, and no longer uses that mysterious 0-299 bit format for the test bit. Now you can specify any bit on any object, using the same format as the rest of the formatted output modes. Here's an example:
console = " DIO0 bit is ^N?4:D0:0^ON^OFF^."
If the DIO0 bit 4 of the first readback value is a 1, this will display:
DIO0 bit is ON.
The conditional string capability has been expanded to include compare functions against a constant. For example, instead of checking a bit, you can compare a variable or timer against a value and display one of two strings depending on whether the condition is met:
console = " Timer 56 status ^T>100:56^has timed out past 100^is still active^."
See the updated HCS_C_Compiler Users Guide for more info on conditional strings.
8/15/2008: Here's a screen shot from 1.09 using the new and improved host executable (soon to be replaced by a Visual Studio 2008 version!!). I've tested with several combinations of different HCSNet cards and a lot of different bit/hex/decimal/char formats. While the HCS_C itself can handle up to 16 HCSNet cards of each type (Combo, Mini, DIO, TERM, Image), this version of host can only show devices 0-3 of each type (soon to be increased). I can't do strings within a string, but this example shows the character display escape sequence within a string from the DIO0 device using this escape sequence in the string going to the console:
console = "HCSNET DIO = ^NcD0:1;NcD0:0^ ^n7,8^"
(the DIO0 window itself just shows the received string in the order received, so in this case it looks backwards). And isn't it nice to have a working <CR> within the console now! <CR> is that escape code ^n7^, the ^n8^ code (commas now can separate a string of specials) moves the cursor to the right one space. You can also do cursor positioning and screen clear within the console window now--and of course the host display is now much bigger. A new feature for 1.09 is the "ALERT" capability, if you do an events action called ALERT(n), this ALERT window automatically pops up on the host display. You can pulse the ALERT on and off in your events file to make it flash. Before too long, I will have a string you can specify at any time in the events file (it can change for the same alert!) that will attach to each alert that is active, but for now 1.09 will just show the hex code for the 16 alerts. Oh--and you can create defines for these escape codes to make them more readable--in fact, I may create some predefined ones before releasing 1.09.
8/10/2008: Well, 1.09 is just about ready to go out, barring anything late-breaking that I find. I'm going to initiate a final round of testing on my inservice HCS_C for reliability, then here it comes. Many, many changes, so be prepared for issues before you decide to make the update. Historically updates of this magnitude go through about 2 iterations before it ends up being fairly clean--this one may be more because of the extent of the compiler and host changes. I'll put out a console port user interface soon so you can start making your own custom apps to talk to HCS_C.
8/7/2008: More updates for the HCS_C_Compiler Users Guide. Now that the % escape is gone, there's no define substitutions for variables within a string (and no <string>, variable list format), which leaves no way to specify a defined name for variables in a string. This is very handy and needed to help keep track of which bits are connected to what ports, and simplifies changing a port pin without having to change all the corresponding references in an events file--so I'm going to add the "tick-define" capability. You can define names just like before, but now you can actually do a substitution within a quoted string using tick defines. See the HCS_C_Compiler Users Guide for examples. I also added a "UPDATED" keyword to allow you to quickly see all the new stuff added to this document.
8/5/2008: I created a very nice regression test that covers all of the display modes, so hopefully you wont get any surprises when you use the new 1.09 code (but be prepared anyway, this is a very big update with a lot of fixes in a lot of places). It works much better than the old display mode formatting, now the right and left justified cases look like they work correctly, and the signed decimal display is new, and the over-range value display is pretty cool! 1.09 is just about set to go, I'm running a bunch of last minute tests... You will need a new compiler, host program, and firmware to use 1.09. This will be so nice, because I kept running into problems trying to incorporate the old stuff while turning on the network card--that's all gone now, so network card turn-on and webpage generation should be much easier now.
8/3/2008: I updated the HCS_C_Compiler Users Guide to give you a taste of what is coming with the new string escape sequence syntax. If the display mode selects a number of digits insufficient to display the value of the object, HCS_Xpress now shows asterisks rather than a max value which could be misinterpreted as a legitimate value.
8/2/2008: Still in heavy testing on 1.09, still finding a lot of bugs--but it's going to be a significant improvement. There are two major updates in 1.09: First, the HCSNET support is much better (which aint saying much, unfortunately--but now it readily supports any collection of remote devices), and secondly, this is the first version that has the HTML support escape sequences.
Note, as I forewarned a while ago, now the % escape sequences are gone--this code has been completely redone. I think you'll like it, it's now a very consistent syntax that replaces the old hodgepodge of some display objects were only supported by some output formats, and will make HTML web pages much easier to generate--but it has the serious consequence that you will have to replace the old % escape sequences in your existing events files (in the new architecture, it was just too clumsy to support the old hodgepodge!). Now there are 6 classes of objects, and 4 display modes, which any (non-Special) object can use:
OBJECTS: Variable, Register (IO), Timer, Memory, HCSNET, and Specials (Date/Time, conditional string, and video)
DISPLAY MODES: Binary, Decimal, Hexadecimal, and Char
The binary mode supports display of any bit or any set of contiguous bits in the OBJECT.
The decimal mode supports variable, fixed left justify, fixed right justify, and fixed right justify with zero fill (corresponding to the old HCS2 P,Q,R,S display modes). In addition, now decimal will correctly display negative numbers, and if the number of digits specified in a fixed mode are not enough to display the actual value, now a maxed out value is shown. For example, 1234, when displayed in three digit decimal form, will show 999.
The hex mode also supports variable, fixed left justify, fixed right justify, and fixed right justify with zero fill. If the number of digits specified in a fixed mode are not enough to display the actual value, now a maxed out value is shown. For example, 1A54, when displayed in three digit decimal form, will show FFF.
Note that these new modes are available for every device on HCS_C--the display environment is common to every devices that can display a string, including the console, video, and any HCSNET device.
7/28/2008: I'm working hard to get 1.09 out the door. It seems to be working well, but I'm finding lots of cases to fix--the HCSNET interface has been completely redone (I was disappointed, the old version really didn't work very well). Now readback, especially of legacy devices, can examine any byte of a returned string (before you could only see the first byte), and more importantly, now readback of more than one HCSNET device should work correctly. I'm starting developing the architecture for downloading events files over the web since it's a nuisance to have to go to the utility room to check/load new token files.
When 1.09 comes out (I think sometime in August), it looks like you'll be able to pick up the new 3D host app, and if you want, you'll be able to make your own host app, because the serial protocol is quite a bit simpler. I'm having fun with the 3D host app, it ended up being a little harder than I hoped because in order to display updated HCS_C status, I needed dynamic text to wrap onto 3D objects, which doesn't appear to be supported in DirectX 9. So... I wrote the host app to generate texture files on the fly using the updated HCS_C status, and once a second, wrap those texture files on objects. It works--its pretty cool! I could create a ball rolling down a hill, and you could see the HCS_C status updating as it rolls!! As it was, I just created some boxes with constantly updating status on them that you can "walk" around. I'll get a lot fancier later, but the proof of concept definitely works. The big question is just how much computing horsepower is needed to make that practical--I'll try it on my older laptop and see if it still has reasonable performance. But for now, 1.09 has my focus...
Don't worry about the apparent status on the online HCS_C--it's been operating flawlessly for almost 4 months, staying online in spite of a large number of received bad packets, resets, and other junk showing up on ethernet. It currently is in transition as I work on adding sensors, installing and testing 1.09 alpha, and so on. I wish I could say the same for routers--I've tried two different Linksys routers and a Netgear router, all of which lock up about once a day. When I leave home, I need those routers to stay working, but the junk just isn't reliable. I'm going to try loading DD-WRT on one, see if that's any better--otherwise I'm going to have to spend money to get some kind of commercial set up. Everybody I know is complaining about their routers locking up frequently--how do companies get away with this. I'm hoping HCS_C will always do better than this...
7/17/2008: Ahhh!!! Sometimes HCS_C, and all the interesting pathways it takes, is just a source of indescribable fun! It turns out DarkBasic was too slow for my 3D virtual house--so I spent an entire weekend coming up to speed on the Dark GDK, which is free at http://gdk.thegamecreators.com/. You can get it to work with the free Visual Studio C++ Express from Microsoft, along with the Free DirectX SDK, to create C++ programs that are far faster than DarkBasic Pro. One thing--it ONLY works with the Aug 2007 version of Visual Studio, don't get a newer version, it will not work and is a bit ugly to clean up. You can't beat DarkBasic Pro for quick prototyping, but it is performance limited, whereas so far the Visual Studio and Dark GDK, while rather more painful to get the serial port connection working (took hours!), cant be beat for performance (I just couldn't keep DarkBasic Pro from dropping characters, and the serialport plugin wouldn't support the binary transfers that HCS_C does). And now I have learned the ability to create a simple 3D windows app! Stay posted, I'll try to put up a simple demo that should talk to your HCS_C (with only a very limited feature set) that will show you why I'm so excited! I'll put some pictures up shortly--it won't look like much, but hopefully you'll see the potential, and this should make it very easy for any of you to create your own customized host program. Note, you will need 1.09 HCS_C firmware to make the demo work (that version will also require a new 1.09 host.exe that talks the new protocol). I should be putting up an alpha version of 1.09 this weekend or maybe into next week just so you can try out the new host app demo... but be prepared for some major code turmoil, if you don't want to do that, you'll want to stay on the sidelines for another couple of weeks.
What's really great about this, is now the serial port protocol is fairly clean, so you can write your own app (C++, Visual Basic, Linux app, etc) to talk to the serial port. I've removed all the timed round-trip protocol stuff that was in HCS2, so now the interface is very simple--basically, HCS_C continuously sends data in binary format with a device header. You can receive these packets and display them however you want. If you need a particular packet, you can send a command on the serial port, and eventually a packet will come back with the data you want (there's no timed connection, and other packets may arrive before that one, but eventually you will get what you requested).
I'm just about ready to turn on ethernet, cross your fingers--it should go smoothly, I hope...
I was sickened to see DarkBasic cracked, these guys have created an incredible environment for the cost of a computer game. Do us all a favor and buy it, support these guys...
7/13/2008: Well, as I fix the HCSNet support, I have been unable to get the host program to keep up--it's an ancient DOS program that we've been stuck with because it uses a bunch of intrinsic functions provided by the old Borland 3.0 compiler. It's long been time to replace it, but no-one stepped up to the plate, so now I need to do it. I need something much more extensible to handle status for all of the HCSNet cards that I have (4 mini's, 1 combo (probably going to be two, shortly), and the new camera card coming out shortly). In fact, as I said, the host program wont work at all--as I tried to fix the HCSNet stuff so it is cleaner and more capable, I keep running out of memory, it will only compile the whole program and data into 64K. I could change the memory options, but when I tried that before, it never worked. Soooo.... I decided to look at some options.
I have wanted to make a 3D virtual house, placing virtual sensors throughout. The idea was to be able to "walk" through the house, looking at the actual sensors being communicated to the HCS_C main board by clicking on the virtual sensors, and turning on lights, etc, by clicking on "virtual" switches. Since DarkBasic is a relatively easy prototyping language that talks 3D, I gave it a try, and was successful at making a simple display of a portion of the HCS_C status in real time on a very simple host program (in 3D!). Not very impressive, but a huge amount of future potential.
DarkBasic has a serial port plugin DLL that makes easy work of connecting DarkBasic to the HCS_C. It has one drawback, though, it is a DirectX 9.0c wrapper--which means to run this application as host, you have to have a moderately fast machine with a good video chip set. And, I found out, you have to have the very latest DirectX 9.0C. Foolish me, I thought there was only one DirectX 9.0C, but there's a zillion versions of it, you have to get the very latest. And, now the Microsoft site wont let you download it unless you run their WGA program that checks your machine that it has a non-pirated OS--it worked OK, but it was a long effort to get this all working.
That wasn't the end of it, though--I had swapped my desktop computer to a much more powerful one with no external serial ports, but fortunately the motherboard happened to have a standard 10 pin connector for a COM1 port. Much to my annoyance, ASUS used a standard connector but an non-standard pinout, and it's not documented in their user manual, the motherboard website, or anywhere else that I could find!! Serial ports are definitely going away. To keep HCS_C going, I'm going to get a USB to serial port adapter (any recommendations?) but for now, I had to manually figure out the pinout and make a special connector--that took around 6 hours of precious weekend time! arrrg.
Anyway, in the end, I got it working, and then I discovered that the DarkBasic DLL absorbs some special characters, and will only accept 7 bit format, meaning that the 8 bit data that HCS_C sends to the console must be encoded. Arrrg. I quit there, but it was a very productive day--and the potential for a really fun interface (finnnnaaaalllly) for HCS_C is definitely there. Even better, I will standardize and hopefully document the interface so you can write your own host app... but there's a very large amount of work here, I'm time sharing between this and the network card. Fun stuff, but I hope you all don't run out of patience...
7/6/2008: Oops. sorry, false alarm on the wait statement, it's working fine. The host compiler change to accept a larger status word was causing some console statements to be dropped, making it look like the wait statement was broken. But I'm still working on the legacy HCSNet device support...
7/5/2008: Took a short break from the ethernet card to do some HCSNET firmware cleanup. The legacy RS485 query commands have quit working, found several problems, especially when multiple DIO devices attached. I'm redoing the receive string storage, it was a dynamic scheme before, and was too complex with too many corner cases. Instead, I'm going to have a dedicated configuration space in front of the token list, that will make life much easier to get working and test--but it means now 256 bytes, rather than the old HCS2 47 bytes, will be allocated for the config space. Since the the token array is now 4 times bigger (16384 tokens rather than the old HCS2 limit of 4096), I think I can live with that. I'm working on fixes for that... and I seem to have caught a failure case with the WAIT statement. Fixing these, should have new bits before too long.
7/4/2008: Happy Independence day for you USA readers! I've got part of the network card turned on, here's a picture with indicated features. So far, the board is looking good, the FPGA loads up correctly and refreshes the SDRAM. Turns out I got the wrong DRAM pinout, so I had to use some old 256Mbit parts rather than the 512Mbit I intended. The boards have jumper configurable standard IO, you can set each 8 bits to be inputs or outputs. It will be a long time before everything gets turned on, especially the Cirrus and Microchip DSP coprocessor code--so I will upload the Gerber files if you want to make your own. Since ethernet will be brought up first (hopefully wont take very long since this code is already running on the prototype that uses a different FPGA), I probably could have my arm twisted to make you an ethernet only board... But as always, assume everything will take a long time...
6/28/2008: Well, I'm back! And so is the network PC board. It's just getting turned on now, it's got a couple of small mistakes on the powerplane that require a small cut (a via that crossed two powerplanes, I have no idea why the layout design rule checks didn't catch that). But--I'm going to be gone again in a couple of months! However, I'll just be doing what I can when I can. I now have the Avnet eval kit--it doesn't come with any software at all, so it's not as good a deal as I thought for you all to migrate to for the next generation of HCS. I don't know what the answer is, but I do know HCS_C is showing its age and I'm doubting the wisdom of pouring large amounts of efforts, HW and SW, into the current platform. It just takes too long for me to get stuff done--I've already had one customer move on because of that, so I'm thinking that some radical rethinking is needed that is quicker and more supportable by you all.
5/5/2008: Down to the final stretch on PC layout for the network card. I want to get this done before taking my leave shortly, don't know if I'll finish in time. The board is coming nicely and will provide a good migration path for updating the HCS_C main board in the short run. The limitation of 16 bit processing and the need for better display formats made me realize that a better math package for HCS_C would be valuable, so the network card will have a Cirrus 9302 for coprocessing support. It also has a MAC, but I have had such good luck with the Microchip EN28J60 prototype that it will be the ethernet path for now.
NOTE: due to the increasing need for better and more consistent math support, more memory, and a variety of other issues that are coming up, and my ability to manage only one path, HCS_C standalone development has come to a stop. The network card will become a requirement for future revisions of the HCS_C FW. The standalone HCS_C needs other players besides me to carry on the FW! It's not hard, but I have more than I can manage with just one path. Be aware that the old parts on the HCS_C main board are getting hard to find, so it will be replaced before too long with an updated version. Take a look at the Xilinx Virtex-5 development board designed by Avnet for $395: onboard processor, DRAM, ethernet, RS232, Flash, USB. This is a great deal and a nice integrated module for the next generation HCS_CX. Go to the avnet.com site and select the Programmable Logic page, FPGA, then go to the bottom of the Design Center page, it is the AES-V5FXT-EVL30-G evaluation kit.
In summary, the future of HCS_C (at least as I see it, unless someone takes over, or if you all persuade me otherwise!)
1: Thruough 2008: Network card daughtercard, full system now called HCS_CE. FW for HCS_C standalone has stopped at version 1.09. Standalone HCS_C will no longer be shipped (but existing HCS_C boards will work with this ethernet card and new FW versions will support that combination). Estimated ethernet boards back and ready for turn-on and FW development in June, 2008. FW development shouldn't be too bad unless problems require another layout (it's a different FPGA and new processor, so a PC layout spin is probable), you may be able to get boards end of summer or fall 2008, although I would be prepared for considerable instability until the end of 2008.
2a: Targeted for early 2009: Update the HCS_C main board. Two approaches being considered: Re-layout of the HCS_C and ethernet daughtercard into one board called HCS_CX. This approach will be quickest and probably would begin in fall of 2008.
2b: A more forward looking approach is replacing HCS_C altogether with the virtex-5 eval board and an additional add-on PC board with the HCS IO capabilities on it. Since there is a learning curve, new tools, and different processor, this is a big change. Only one or the other of these two options will happen. I'm leaning a little toward the virtex board option.
Send me your opinions and thoughts! One thing for sure--HCS_C is showing its age, and we need to think about its replacement...
4/24/2008: I've been in PC layout for the network card--don't have enough pins on the Xilinx FPGA! Xilinx uses a 208 pin PQFP, whereas Altera uses a PQFP 240. Cheap low cost pcboards and assembly preclude the use of BGAs. My prototype network card has been online for close to a month, so I think I got most of the issues out of that implementation.
4/5/2008: Wow. This is the first time that having an installed HCS_C paid off big time. I've got a network of machines here that I keep running, and this morning, HCS_C was reporting abnormal network activity. I say, what the heck? and found one of my machines was a zombie, emitting thousands of packets continuously. I'm fairly fanatic about safe computing, so I was astonished to see this--AVG antivirus reported no virus, yet clearly the machine was obviously badly misbehaving. HCS_C reported that all the packets were "unknown" type, and would stop as soon as I disabled the offending machine's network connection. I checked all active processes, and found nothing suspicious--but that particular machine is a gaming machine, I don't get email or even do browsing on it (and only play local single player games) so I'm guessing that one of the game updates had a virus attached. There was no recourse but to format the hard drive. Unfortunately, I have no idea what caused the problem (I'm the only user on that machine), but what an enormous comfort to have a powerful unexpected benefit to HCS_C--continuous monitoring of internet activity. Even AVG couldn't catch this one--I've heard that antivirus software can't outsmart the latest breed of viruses, but HCS_C is like having a continuously running version of wireshark checking network behavior!
All is quiet on the network now--but I'm going to immediately put in local and web alarms for abnormal ethernet activity. I'm currently installing several more HCS_Mini and HCS_Combo cards, the Combo card has voice, and HCS_C is going to speak various alerts on our intercom. If the bad activity occurs at night, it will just wait until morning to report so it won't wake us up.