A new era of add-on debugging

The Log Viewer Window

Today marks the release of the DebugLogViewer and LibDebugLogger 1.1. Together these two will provide authors with a powerful new way to find and fix bugs in their add-ons locally and out in the wild.

But the log viewer is not just for authors. It is also very useful for regular players, as it will for example suppress Lua error pop-ups and instead present them in a non-intrusive manner.

LibDebugLogger 1.1

The new version of the library now captures chat output made via the in-game debug functions, error alert messages and loading screen occurrences. It also provides some callbacks so other add-ons like the log viewer can react to modifications of the log data.

LibDebugLogger data as stored in the saved variables file

Any add-on can use LibDebugLogger to create custom log output and users can then send the LibDebugLogger saved variables file to the author when they encounter a problem. The logged data will make identifying the cause of a bug easier than ever.

Check the description to see how to introduce LibDebugLogger into your add-on.

Debug Log Viewer

The Debug Log Viewer consists of three major parts.

Quick Log

The first thing you will notice after installing it, is the new “quick log” window. It will appear in the bottom right corner of your screen and can be configured to show parts or all of the latest entries in the debug log.

The quick log displays logged messages in a small part of the screen

For the convenience of regular players, it is pre-configured to hide entries in the debug category and also those generated by LibDebugLogger itself. As already mentioned, LibDebugLogger will now capture chat debug messages. To avoid having them shown twice, they will be filtered from the chat automatically and instead show in the quick log. Authors can opt to use the new LibChatMessage in order to avoid abusing the chat debug function and instead use proper system messages. That way they will be shown in chat regardless.

Clicking on the time or tag of a message will open a detail view for the selected message. In case a message contains an item link or similar, the link can be clicked like for regular chat messages.

Add-On Settings

The settings menu allows you to configure the quick log, log viewer and LibDebugLogger. You can change the colours used for the different log levels, and more.

Log Viewer

The most important part is the log viewer itself. It is a new window that can be opened via the /logviewer command, a key bind or the context menu of the quick log.

It shows all log entries in a list and allows to filter them by several criteria. When an entry is clicked, it will open a pop-up with the full formatted output and – in case the setting is activated in LibDebugLogger – it will also show the stack trace for the message. The list can be filtered by log level, time frame and entry tags.

A Log Detail Pop-Up

If you like what I do, consider supporting me on Patreon. There are still many more projects I want to realize and your patronage allows me to spend more time working on them.

How to get ready for a major game update

Keep Calm

You may be excited and want to jump into your new adventures as soon as possible, but you should restrain yourself and first get some things out of the way. Otherwise you may ruin your own experience.

Update The Game

Arguably the most important step on this list. Just start the game launcher and let it do it’s magic. Be aware that the first login after a major update always takes longer, because the game has to rebuild some internal caches.

Create Backups

Once the game has received it’s shiny new files you may want to create a back up of the whole game folder. In case you have enough space available, this can save yourself a lot of time should one of the incremental patches in the following weeks fail to install correctly. Worst case a failed update could require you to re-download the whole game, which is already over 78GB and can take hours. Copying a backup is likely much faster, but the decision is yours to make.

More importantly you should always create a backup of your User Folder. This folder contains all your add-ons and local settings. In case something goes wrong, this can save you from having to redo your add-on configuration.

Redo your UserSettings.txt

This is an optional step you can take to potentially improve your game performance and in-game experience. Major updates sometimes bring new settings or better default values with it. Letting the game recreate this file in your User Folder means that those values will reset properly, but as a result require you to redo your video settings among some other things.

If you are tech-savvy enough and know what you are doing, you can also use a merge tool to copy over some of the old settings after you let the game create the new file and closed it again.

Update All Add-Ons

Major game updates always bring some major changes to parts of the add-on API. This means some add-ons require an update in order to be able to run on the new game version. Before you log in after a major update, you should update all your add-ons using Minion or from hand.

Be aware that there is currently a transition happening and authors are unbundling libraries used in their add-ons so they can be updated separately. You should check the change log in order to see if you need to install some new dependencies, otherwise some add-ons may not be able to load.

Allow Out Of Date Addons

In case some add-ons have not been updated yet, you may want to check this checkbox in the add-on menu in order to allow the old version to load. This is will disable the API version check and simply run all add-ons on the current game version.

In case you encounter some errors after log in, you should post them on the esoui forums, or if you know where it is coming from, directly in the comment section for that add-on.

So much to do, so little time

TL;DR: Many things planned. Become a patron so I can get them done sooner.

The Elder Scrolls Online has recently turned 5 years old and it looks like a bright future is still ahead of us. New players and add-on authors are flocking to the game and there is no end in sight. That’s why I decided to rectify some of the past mistakes we made and also plan to create some tools that will help authors become more productive and save them countless hours while they work on their add-ons.

The following projects are a non-exhaustive excerpt of my to-do list in no particular order.

LibStub Deprecation

Originally ESO didn’t differentiate between regular add-ons and libraries. That’s why the authors that were around during beta decided to port LibStub over from WOW and include library code as part of other add-on’s source code. This unfortunately means that an arbitrary number of older versions of the library code could be executed on login. In cases were an old version includes a bug in its initialization code, it would also affect all future versions. Which did happen on several occasions and would either require all authors to update the library included in their add-ons, or users to manually delete the faulty versions in case they used add-ons that haven’t been updated.

I made my first attempt to change this back in 2016, but it didn’t gain much traction and was quickly forgotten. Ever since I have secretly continued to pave the way and my plan finally started to come together last year after ZOS has introduced a feature to let the game handle versioning of libraries and load them from folders nested inside other add-ons. That way only the newest code will run and bugs in older version won’t cause any more problems. It also opens doors for libraries, as they are now basically the same as regular add-ons and can use all the same features that are available to them (XML files, named UI controls, saved variables, dependencies etc.).

Starting with Elsweyr, libraries will also be shown in a separate list at the bottom of the add-on manager if they specify that they are indeed a library.

Debug/Chat messages

Over the years, most add-on authors (including myself) have been abusing debug functions to show messages in chat. This caused more than one incident where debug messages where accidentally left in a published add-on. For this reason I have created two new libraries and an add-on to allow authors to properly separate debug messages from regular chat messages and inspect them.

LibChatMessage will offer a simple and clean interface for printing messages in chat. It also has options to keep a history of chat messages and prefix them with timestamps. These are two features that other add-ons like pChat already offer, so I decided to disable them by default and you will have to configure them yourself if you wish to use them.

LibDebugLogger is already released and stores a lot of useful information that can be used by authors to determine why a problem occurs. The next version will include additional details, like output made via the in-game debug utilities and certain game alert messages. Sending this file to an author when reporting a problem will hopefully go without saying in the future.

DebugLogViewer will act as a frontend for LibDebugLogger and can be used by authors and players to view and control which messages are shown. When it is active, it will also suppress the in-game error pop-up and chat debug output, as those will be displayed in a less intrusive way.

Authors are advised to check them out once they are released and decide for themselves if they want to switch to LibChatMessage for their chat messages, or leave it as is and have it redirected by the DebugLogViewer.

LibAddonMenu 3

The current LibAddonMenu has a few limitations that authors have complained about ever since I took it over from Seerah. In order to fix them, it is necessary to rewrite core parts of the library, which is on my list since 2015. There are also some cool new features (setting profiles, sub panels, tabs, dynamic controls) I wish to implement, that will give authors more flexibility in how their menus are structured and make it easier for players to configure their add-ons. Aside from that, the documentation is also in dire need of an overhaul.

Minion 4

Seeing how Minion hasn’t received any notable updates in quite some time, at the end of last year I decided to take matters into my own hands. Thankfully Dolby agreed to accept my help and since then I have been working on and off on fixing bugs and adding new features to it. One feature I am especially excited about is the introduction of add-on dependency handling. Minion 4 will be able to recognize and automatically install libraries that are required for an add-on to run.

In addition, Minion will become open source once the new version is up and running. That way other interested parties will be able to help improve and fix it going forward and hopefully keep it in good shape for many years. It will still take some time to finish everything, but rest assured that it is coming!

VSCode ESOLua Extension

For the longest time I have been using Eclipse to develop my add-ons and also created an autocompletion for ESO to be used with it. Sadly the LDT plug-in used to write Lua code has been abandoned a while ago and the autocompletion is hitting its limits too, due to the growing size of the game API.

Since Microsoft announced that they will finally bring support for detachable workbench parts to VSCode in 2019, I plan to switch over and create an extension that will provide all the features an add-on author can dream of: full autocompletion, code validation, refactoring utilities and whatever else will be useful. It will of course take some time to cover all of these, but in the long run it will save a lot more and make developing add-ons more enjoyable.

Kagrenac CLI

This tool will simplify a variety of common tasks that all add-on authors need to do in order to release their add-ons. Once completed, Kagrenac will be able to prepare projects from templates, validate the add-on structure to prevent some common mistakes, create zip archives and upload them to esoui. This will also be a big time saver for authors.

ESOUI Inspector

There are several developer add-ons that allow us to inspect one aspect or another of the game, but they all have some shortcomings. My plan is to revamp my sidTools into something that works similar to Chrome Dev Tools and gives authors a more holistic and structured approach to inspecting the game API and GUI.

Simple Notebook Tutorial 2019

A complete overhaul of the tutorial to bring it into the present and make use of changes that happened since I initially wrote it. I’d also like to fix some parts that have raised more questions than they answered and finish the missing chapters so it is finally complete.

Automatic Add-on Updates

I have left some of my own add-ons to collect quite some dust, but wish to change this and provide a new version for each major game update semi-automatically. This will become easily possible once Kagrenac is ready to upload new versions to esoui. Once I have finished to set this up, I’d also like to write a blog post on how I did it, so other authors can do the same.

AwesomeGuildStore

It has been in limbo for some time now, while I have been busy with other projects. Once Elsweyr is released, I plan to get it back on track. There are some changes in the game that will allow me to fix the item preview and I will also investigate ways to reduce the impact of server request timeouts, so I can finally get rid of the beta label.

As you can see, I will be pretty busy for quite a while. Most of these projects focus on providing tools for myself and other authors that will save a lot of time in the long run and make developing add-ons easier and more enjoyable for newcomers and veterans alike, but I promise I will also give my existing add-ons some love and update them.

If you want to support me in my endeavours, consider becoming a patron so I can spend more time working on these projects.

Schema definition for ESOUI XML

Creating GUI in Elder Scrolls Online is time consuming and frustrating. For a long time I have played with the thought of making it somehow easier and finally decided to look into that.

After thinking about it very hard for a few evenings and researching different ways to improve the experience, I arrived at the conclusion that it would be best to create a schema for the XML files used to define the GUI elements. This way I would learn more about the available possibilities and end up with something that will allow me to craft them faster and with less errors.

The first step was to learn how to define a schema for XML files. After a bit of searching around, it turned out that there are two ways how it can be done: Document Type Definition (DTD) and XML Schema Definition (XSD). Looking up the differences, it quickly became clear that DTD is basically obsolete and XSD was the way to go.

Now it was time to find some resources on how to write actual XSD files. This turned out a bit harder than expected. For some reason all the results I found where copied parts of the same tutorial, which seems to have originated from w3schools. They cover the basics, but don’t really explain much in depth. With their guides and googling some StackOverflow answers, I managed to write my first schema that would validate the first few elements in an ESOUI XML file!

But this was only just the beginning. The next hurdle was to figure out how to model the actual content of the ESOUI XML. For that I had to take a deep look at the “UI XML Layout” section in the ESOUIDocumentation and figure out how all the things there are related to each other. After a few evenings of thinking and doing, I managed to write an XSD that could properly validate a couple of files from the ESOUI source.

It became quite clear how everything is structured and that I only needed to repeat a few templates in order to create a complete version. But that would be boring to do by hand, wouldn’t it? Especially since the XML could change at any major update and new attributes and elements could become available or old ones could get deleted. That’s why I decided to generate the XSD automatically from the documentation file.

I already had set up a JavaScript to parse parts of the documentation in order to generate an Execution Environment for Eclipse LDT in the past, but left out the section about the UI, since it was not necessary back then. After some tinkering I managed to parse the remainder of it and generate the first full version of the schema. My Windows had Visual Studio set as default program for .xsd files, so I decided to take a look.

Turned out VS can visualize the relations between the controls in a neat way.

The next step was to validate all files in the ESOUI source to make sure everything worked, but opening the xsd file in Eclipse for the first time turned out differently than expected and I was greeted by a lot of errors in the part where I defined the different Control Types.

cos-nonambig: Anchor and Anchor (or elements from their substitution group) violate “Unique Particle Attribution”. During validation against this schema, ambiguity would be created for those two particles.

After some searching around and sleeping over it, it turned out that the way I did the inheritance of Control and AnimationBase elements was not going to work, since it would add the same group of elements to both the parent and the child type. So I had to introduce a virtual base type for each of them and handle the actual Control and AnimationBase as regular child elements.

Since the first version of the script was just hacked together and didn’t really allow for an easy fix, I decided to use this occasion to learn something new and rewrote everything in TypeScript.

Now Eclipse also accepted the schema, but even after an extensive search I could not find a way to globally set it to be used for all xml files in a specific directory. So instead I decided to just replace the opening tag in all xml files to include a noNamespaceSchemaLocation.

The result was a surprisingly short list. It reported 36 errors, most of which are typos or misplaced attributes.

The final part was to create a second schema for the bindings xml files, but since they are not part of the documentation and didn’t look very complicated, I just created a schema based on the available code in the ESOUI source.

This first version of the schema already works quite well and the autocomplete is awesome, but I have some plans to make it even better in the future. One thing I will add for sure are more specific rules for types. Right now most attributes are just strings, but the name attributes for example support variable expansion for things like $(parent), so I would like to make sure that these are completed and validated correctly too.

The schema files for the game version on live are available via the following two URLs:

In order to use them in Eclipse you just need to specify them in the root element of the xml like this:

This will make Eclipse download the xsd file automatically. If you don’t want to rely on an external resource, you can also set a local override file in the settings. Just go to XML > XML Catalog and create a new entry with type URI, where the key is the exact link from above. This can also be used to specify a different version of the schema, e.g. for PTS.

You can also find the tool I wrote to convert the ESOUIDocumentation.txt on github.

 

Welcome!

Welcome to my Blog! This is where I will write about my programming adventures and maybe some other things.