[bug] [fixed] Ethernet on Labrador 64bit , package loss, no ssh, high latency

Labrador
#1

Just trying again to use Labrador for my IoT framework, got a new 64bit Labrador version with an oral promise that Ethernet should work now - and I was hoping that cross compilation for microcontrollers with platformio as well as wifi ap mode might work now.

Out of the box, there was no (internal) ethernet at all, so I tried the 64 bit installation from here https://wiki.caninosloucos.org.br/index.php/Instalando_um_sistema

The “new” image boots and does support ethernet but at really low speeds and horrible latency - turns out that the new image seems to downgrade the installed kernel 5 to 4.19.37.

The installation also initially destroys the Labrador as the install script is buggy (more in another thread).

Where can I find the glorified new kernel 5 for Labrador 64bit and how can I install it?

#2

@ulno, hi!

About the 64bit labrador, the ethernet really works fine for me, i did a lot tests, but i found a bug that is very close of your problem, but it only happens when using the old base-boards like 1.0a, for while ethernet should work fine only with base-boads v2.1. I will add some section in wiki to describe as bug, it will be fix later if only affect old base-boards.

What is the version of your base-board and core-boad?

about this, we are only using kernel 5 in developer mode, it works, but there are no release version of kernel 5, our official version is found in kernel 4.19 64bit that shoud be the same in the inside of image lab64_v0.2.2.img.7z

That is bad, i will check it. But could you be more specific? what happens?

For while you will not find it anywhere, as soon as we let it avaliable i will do some post in forum.
We probably only will release the kernel 5 with gpu driver working.

Best regards

Igor Ruschi

#3

I have baseboard v2.1 and core v3.1.

It’s like only some messages go through. I have pings in my local network close to 800ms and transfer rates in the Kilobyte range. There are no errors in dmesg.

I am not interested in the GPU at all (at the moment) as we (Roseli Lopes and I) want to see if we can use the Labrador as an edge gateway for your IoT classes.

#4

I will take a look, as soon as i had news i will report here.

#5

@ulno, Hi.

I trying to understand what happens, but i can’t reproduce your problem using any baseboard v2.1 in combo with core v3.1. It only happens when i made the combo baseboard v1.0a with coreboard v3.1.

I had done tests with 4 combos between baseboards and coreboards, but anyway i had try it works fine. I get download range up to 10mbps, i its too close of limit to a ethernet 10/100 as we have, so i can’t complain of speed.

Screenshot%20at%202020-01-24%2011-32-53
Screenshot at 2020-01-24 11-32-53.png1920×1080 204 KB

For all tests i had done, i was using the img provided lab64_vxx.img.7z.

I had use qbitorrent and download the debian img, this ensure a great download bandwidth.

Testing the ping time, i got around 20ms to the gatway, that is bad, it shoud be less than 1ms in my opinion. But a slow ping doesn’t means a bad speed at all. It only affect a low latency applications, like games.

Are you sure that do not use the combo with baseboad 1.0a and coreboard 3,1?, please try to make sure what baseboard you are using, and try to reinstall the system.

In any case, could you give the output of the following commands?

$sudo ifconfig

$uname -a

$lsmod

And if you still have the manufacture number(white paper with code like AD00XX) of your boards, inform them too.

Casimiro, found some unexpected behavior in ethernet too, but i don’t believe that is related with yours…

Best regards

Igor Ruschi

.

#6

I am traveling for the week in Germany), so I have a different test setup here now.
My (Linux-)laptop is connected to the local WiFi.

=== tests from Linux laptop ===
With speedtest.net I measure from the desktop directly via WiFi to the internet 25MBit down and 5 MBit upstream (speed ramps up nearly instantaneously).

Pinging Google results in an 10ms average.
Pinging my internet host ulno.net in France gives an average 27ms ping.

=== Labrador connected directly to shared ethernet connection on Linux Laptop ===
Pinging directly the laptop yields an average of 15ms but a couple of >1000ms and a package loss of 7%.
Pinging google yields an average of 17ms, a couple of >1000ms and a package loss of 3%.
Pinging ulno.net has usually more than 1000ms (!!!), sometimes a few with 500ms, and a package loss over 5% - is something with the MTU messed up?
speedtest.net though does show 25MBit down and 5 MBit up, but takes a long time (several seconds) to ramp up.

== general ==
Interactive ssh from my laptop into the Labrador is completely sluggish and worse than an old modem connection -> unusable - maybe that would be the easiest to verify, if you can login into the Labrador via ssh on Ethernet and interact without latency in the shell.
Also using apt update/apt upgrade feels extremely slow (that’s where I got my initial values from).

The white sticker on the baseboard shows AD0078 (yes it’s version v2.1) and on the core board AC0806 (yes, version is v3.1).

Using a USB RTL8152 Ethernet-adapter, everything works fast and easily and just adds a ms compared to the laptop.

Hope this helps tracing things down.

ulno

#7

Thanks, this give me some way to follow, for while i don’t believe that has some thing about MTU, it should work with the default MTU(1500).

The lost of package should not happen, it will help to trace, thanks

Best regards,

Igor Ruschi

#8

Were you able to confirm the bad performance of ssh via a peer to peer Ethernet connection?

#9

I have the same issues with my own kernel.

Here the output of the commands you asked, sorry didn’t read it at that point:

caninos@caninos:~$ sudo ifconfig
[sudo] password for caninos: 
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.25.219  netmask 255.255.255.0  broadcast 192.168.25.255
        inet6 fe80::bb18:3a0d:2b62:bdb4  prefixlen 64  scopeid 0x20<link>
        ether ca:06:7d:a8:50:64  txqueuelen 1000  (Ethernet)
        RX packets 607  bytes 58008 (56.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 243  bytes 25862 (25.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 172  base 0x4000  

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 4  bytes 156 (156.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 156 (156.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

caninos@caninos:~$ sudo uname -a
Linux caninos 4.19.37 #3 SMP Sun Feb 2 16:32:11 -03 2020 aarch64 GNU/Linux
caninos@caninos:~$ sudo lsmod
Module                  Size  Used by
bnep                   20480  2
hci_uart               40960  1
bluetooth             425984  23 hci_uart,bnep
ecdh_generic           24576  2 bluetooth
r8723bs               659456  0
cfg80211              671744  1 r8723bs
realtek                20480  1
dwmac_caninos          16384  0
stmmac_platform        20480  1 dwmac_caninos
stmmac                147456  2 stmmac_platform,dwmac_caninos
of_mdio                16384  3 stmmac_platform,stmmac
fixed_phy              16384  1 of_mdio
libphy                 73728  4 of_mdio,realtek,stmmac,fixed_phy
aotg                   69632  0
ip_tables              28672  0
x_tables               40960  1 ip_tables
caninos@caninos:~$ cat /proc/version
Linux version 4.19.37 (ulno@ulno-t440p) (gcc version 9.2.0 (GCC)) #3 SMP Sun Feb 2 16:32:11 -03 2020

It took about 10s per command to react (I tried to get them via ssh to easily copy them).

#10

Hi, @ulno, i did not try yet the ssh connection, i will do that today, thanks for sharing your configs. i found some strange behavior of the hardware, some features registers are different of datasheet, the register says that the GMAC DMA has more features than described on datasheet. I disabled all these features, but nothing changes.

I’m will keep working on it…

Best Regards,

Igor Ruschi

#11

@ulno, hi!

Good news, i found a serious bug at ethernet, the gmac irq number was wrong, i believe that it should be fine now. Unfortunatly the problem with the older bases still there.

It solved the Casimiro’s bug, and the ping problem, i can do some fine tunes on the driver parameters yet. I seen some packeges drop, so i will take a look at this, could you try to change your kernel to the compiled one inside of output folder? (GIT)

I believe that should solve your problem too…

Best Regards,

Igor Ruschi

1 Curtida
#12

I can confirm that Ethernet runs much better now on Labrador 64 bit. ssh is usable, latency seems to be down, as well as package loss. Thanks!

1 Curtida
#13

Thanks for sharing your results!

Best Regards,

Igor Ruschi