@mike, if you give me a minimum failing case I can try to reproduce the problem and trace through the firmware.
I haven’t been able to do as much development on my own project as I’d like, but I haven’t run into any problems since v0.15–presumably I’m not stressing the same part of the system.
I’ll try to find a simple consistent case.
Can you put up binaries for 15? or tell me what to check out?
Okay, I’ve created a test that shows at least some of the issues.
Here’s the code: Test.java (5.4 KB)
Each test colors in all the buttons, both the “pads” (the center 8x16) and the surrounding buttons. The test has two parameters, the “delay” (milliseconds), which I wait after every single button is illuminated, and the “start delay”, which is a longer pause before I do a full round of coloring. Both can make a difference. Each pair of tests with the same delay is preceded by an initialize() (turn off all buttons) and a 1sec pause.
Each test run consists of 8 tests, with delay of [0, 0, 5, 5, 10, 10, 15, 15]. I do a test run with a start delay of 0, and a start delay of 1000. so it’s easier to see what’s going on, the tests are colored, in order: red, orange, yellow, green, blue, magenta, white, gray.
In the code, the setButton() and setPad() function calls just send the appropriate MIDI notes to the controller. Log.delay() is just a call to Thread.sleep(); if delay=0, sleep isn’t called at all.
Here’s a video:
On the first set, with start delay=0, we don’t even see the first 4 tests (red, orange, yellow, green; delay=0 and delay=5) at all. The first thing we see is blue. The last 4 (blue, magenta, white, gray) all work fine, with a few stuck buttons.
In the second set, we do see red and yellow, but no orange or green. Red and yellow are not fully cleared. The rest work fine, with some stuck buttons.
There are some other combos to test, but this is a start. If you want me to compile a copy so you can run it yourself, I can do that, but we’ll need to adapt it for whatever device you’ll run against.
Hi @mike
I will keep trying to figure out what’s happening.
Thanks for sharing and taking the time to tell us what’s going on and making the test.
If you can make an executable for windows and/or mac, it’ll be useful to reproduce the same exact conditions.
I’ll adapt the controller I have here to the Hachi configuration.
I’ll also see if I can create a MIDI file that will show the issues. have you tried something silly like sending 128 note-on messages at the same time? 
I tested with a bunch of MIDI messages when I was working on this issue, before releasing v0.14.
I don’t remember if I did 128, but 96 for sure, since that’s the amount of buttons I have in my test controller (Turn).
I am currently working on having the MIDI DIN5 available as an USB output and it’s taking quite a bunch of my time (reading USB-MIDI spec to find why it’s not working as it’s supposed to 
)
Tomorrow I’ll set some time to test again this problem, as close as your test as I can
thank you for tackling the DIN midi output! that’s awesome
I’m attaching a set of midi files exported from Live clips that show some issues. you’ll need to loop the midi in each file. I also included the Live file. if you have it, that’s easier.
hachi2.zip (52.6 KB)
ch1-a and ch1-b should work fine, turning on all 128 buttons on ch1. if you run ch2-a with them, you’ll see it doesn’t interfere with ch1-a (though the ch1 buttons visibly lag behind). but ch2-b with ch1-b causes some issues where half of the ch1 buttons are different from the other half. on mine, the first half of the buttons alternate between two of the colors, and the other half between the other 2. weird.
with ch1-c, ch1-d, and ch1-e, you can see what happens when things get too fast. ch1-c works fine. ch1-d, where the note off is too close to the next note on, the buttons just stay on one color. ch1-e, where the notes are too short, the buttons never turn on at all.
I did a version of ch1-d but with fewer notes (like 30), and it was able to keep up even with no gap between the notes. but even with only 30, ch1-e still didn’t work – the buttons never turn on if the note off is too soon after the note on.
I should add I was running those at 100 bpm.
Here’s a jar you should be able to run to run my test
java -cp yaeltex-test.jar net.perkowitz.issho.controller.yaeltex.Test hachi
where “hachi” is the name of the controller you want to test on. it should run through the set of tests I described above.
It’s too large to upload, so you can grab it here: http://perkowitz.net/yaeltex-test.jar
Hi @mike!
I am sorry I haven’t posted in a while.
I have been testing a lot these days, and I thank you for your test, this helps a lot to make sure we’re testing as similar as possible.
With v0.14 binaries, I can see something similar to what I see in your video with the stucked pixels. With v0.15 I don’t see anymore of these.
This is a v0.15 sneak peek (we’re really close to release it) with your modified color table:
hachi-0.15.bin (main!) (79.4 KB)
Then let’s see about the color changes and the delays.
With all the tests (ours and yours), something that comes up is that we are sending a lot of messages really fast, and way over the MIDI spec limit, which is around 1000 messages/second. We are using USB and it allows for faster rates, but we’re still over the standard limit.
Another thing to consider, is that our firmware waits 10ms for new messages before showing the updates.
If you send a lot of messages with no delay, it won’t show anything until this time with no messages is elapsed.
We will call message “sets” to a whole update to the controller (setting to OFF, or to a specific color).
We detected on your first four tests, with delay=0 and delay=5, that the message sets are being sent without a delay between them.
This means that the ON messages are sent with velocity 101, then right after the ON messages with velocity 103 and then again right after the OFF messages.
You don’t see anything turn on on these tests, because the controller receives messages all the time, and the last messages are the OFF commands. This explains the consistency that you mentioned.
If you place a small delay (more than 10ms, but I’d suggest 30-40ms) in between sets of messages, you should see these tests work alright.
You shouldn’t need to place a delay between EACH message for things to work, though if you were working with a DIN interface, a 1ms delay between each message would be needed to comply with the MIDI bandwith.
To summarize, try the 0.15 firmware and always leave some time after sending a set of messages.
Looking forward to your comments!
Thanks for the info. this explains a lot! I will try the 15 firmware and add 30-40ms pauses between each test. assuming that works, what I will probably do in my code is make it so that I can group a bunch of updates together and then add one pause at the end, like flushing a buffer.
by the way, did you implement running status? if I can send it correctly, it might save some bandwidth, especially since everything I’m doing with the controller (aside from the encoders) is just note on/off. or maybe we should have a sysex message where I send a color and a bit-packed list of buttons
I’ll try to do some tests tonight and report back!
Running status is off, but you can change that!
I haven’t really tested it, but in ytx-controller.ino, around line 192 you have the following:
struct MySettings : public midi::DefaultSettings
{
static const bool Use1ByteParsing = false;
static const unsigned SysExMaxSize = 1024; // Accept SysEx messages.
static const bool UseRunningStatus = false;
};
Set the UseRunningStatus variable to ‘true’ and give it a go!
Okay. First experiment. I updated to 0.15, no issues there.
My new test does this:
- Clear the controller, then color all pads pink
- sleep(2 seconds)
- For each delay value [0, 20, 40, 60, 80] millis
- Repeat 25 times:
- color all pads
- sleep(delay)
- sleep(2 seconds)
It cycles through a set of colors, and each cycle of 25 ends on a different color: red, orange, yellow, etc. With the pause between cycles, you’d expect to see the pads end up on each of those colors, even if the color cycle is skipped because of the 10ms delay in the controller. once the delay exceeds 40ms, you’d expect to see it rapidly cycling through the colors.
However, that’s not what happens. I see:
That’s typical, but one time there was less yellow than orange. Another, it finished all green.
So it seems that it still sometimes drops notes (when they overwhelm the bandwidth, I guess), and that the wait time necessary to overcome the controller’s 10ms delay is more like 60-80ms.
Here’s the jar: yaeltex-test-1109.jar
As you’d expect, if I do the same test, but add in a 1ms delay after each note, you see most of the colors. The between-run delay plus the cumulative delays of 128 notes is enough wait time to see most things properly.
Interestingly, if I add a short 1-2ms delay after each row, I see some very blocky colors, where different rows end up different colors seemingly at random. I think that 10ms clock falls in the middle of my 128-note refreshes, and I end up with a patchwork of notes caught and missed. if I raise the row delay to 10-20ms, it mostly works as expected, but still ends up with different colored rows during the pauses. so it’s still losing some notes even at that speed. up to 40ms, and then it keeps up with everything, but of course it’s slow enough to see it running through each color.
Hi again
Try this one: hachi-0.15-faster-midi.bin (78.5 KB)
I bumped up the rate of the timer that reads MIDI.
Tell me if the test improves your test.
On my side, with 96 buttons, I see the colors cycle from the 20ms delay test, and they all finish with the correct color
Interesting… they all turn pink, they all turn red, some or all turn orange… and then it goes into the rainbow animation. but then it comes back for the last bit of the test (sometimes). the rainbow happens every time though, when it should be turning yellow.
Mmm… maybe something I changed, but weird bhaviour nonetheless…
Here’s the same with that rolled back: hachi-0.15-faster-midi-WD-rollback.bin (78.5 KB)
Hi!
I got the same behaviour! But not every time, it happened twice so far.
It happens because of the fix for this topic: Controller sometimes not powering up?
We implemented a timer that if it isn’t reset within one second, it will trigger a reboot.
This, in combination with me trying to make the MIDI reception faster, is making the controller be hung up on the MIDI reception for too long and thus rebooting the controller.
Got to fine-tune this reboot-time / MIDI reception speed set-up.
This one had 1 second reboot time + 5Khz MIDI reception.
This one has 1.5 second reboot time + 3.5Khz MIDI reception:hachi-0.15-3.5k-1.5-sec-reboot.bin (78.3 KB)
Whenever you can, please test them and tell me if it still reboots.
Both the rollback and the last version cause rainbow rebooting on the yellow test (3rd test, 40ms delay). I tried loading the rollback version twice, but same thing.
Mike,
We’ll take this to a support ticket and maybe arrange a call to keep looking into this together.