Jump to content
tschak909

Anyone want to help whiddling down #FujiNet ram usage?

Recommended Posts

I am currently running the #FujiNet firmware through the PlatformIO inspector, and have noticed we're overflowing a bit on RAM usage. I'm currently investigating, and if anyone wants to jump in and try to whiddle it down with me, have at it!

 

image.thumb.png.ab338a214d67102696b434e3dddfb0cd.png

 

The code is, of course, here: https://github.com/FujiNetWIFI/fujinet-platformio

Share this post


Link to post
Share on other sites
Posted (edited)
On 7/25/2021 at 12:05 PM, tschak909 said:

I am currently running the #FujiNet firmware through the PlatformIO inspector, and have noticed we're overflowing a bit on RAM usage. I'm currently investigating, and if anyone wants to jump in and try to whiddle it down with me, have at it!

 

image.thumb.png.ab338a214d67102696b434e3dddfb0cd.png

 

The code is, of course, here: https://github.com/FujiNetWIFI/fujinet-platformio

This kind of thing, memory usage, also performance and random bugs, is actually more difficult to work on than adding new features -- unless there's some kind of 'big stupid' to find, but that's luck, a large part.

Edited by cathrynm
  • Like 1

Share this post


Link to post
Share on other sites

One of the big problems with memory usage is Bluetooth functionality which uses a HUGE amount. The newer esp-idf toolchains won't even build the current firmware due to the ram use. We have been discussing removing bluetooth support for the time being.

 

I had an idea that may or may not work by disabling BT at boot but you can turn on BT mode which would force a reboot into BT only mode. Another reset would return FujiNet to normal/wifi mode with disabled BT. This is the function that disables the BT stack:

 

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/bluetooth/controller_vhci.html#_CPPv229esp_bt_controller_mem_release13esp_bt_mode_t

  • Like 1

Share this post


Link to post
Share on other sites

am I reading correctly that blue tooth low energy mode doesn't use all that memory like classic mode does?

Share this post


Link to post
Share on other sites
15 minutes ago, _The Doctor__ said:

am I reading correctly that blue tooth low energy mode doesn't use all that memory like classic mode does?

somewhat, but the target device has to support LE, which in practice is actually a crap shoot. 

In production practice, ESP32 use of bluetooth is primarily used as a bootstrap for getting a device onto a WiFi network. Our use case is somewhat unusual.

 

-Thom

Share this post


Link to post
Share on other sites

ah so maybe BLE can still be used for the bootstrap and then we can have wifi and limited bluetooth and only if it's still not enough we would be shutting off bluetooth entirely as it can only come back online after a reboot. In the last case of complete shutoff I wondered if loss of ssid or changing id would fail, requiring a fuji reboot or not. That's why I pondered just using BLE for the wifi strap and functions as it uses less memory and provides some functionality.

Share this post


Link to post
Share on other sites
1 hour ago, _The Doctor__ said:

ah so maybe BLE can still be used for the bootstrap

The only thing we use Bluetooth for is SIO2BT functionality (BT Classic serial port protocol). We do not use it for wifi bootstrapping. 

Share this post


Link to post
Share on other sites
On 8/3/2021 at 9:30 AM, mozzwald said:

One of the big problems with memory usage is Bluetooth functionality which uses a HUGE amount. The newer esp-idf toolchains won't even build the current firmware due to the ram use. We have been discussing removing bluetooth support for the time being.

 

I had an idea that may or may not work by disabling BT at boot but you can turn on BT mode which would force a reboot into BT only mode. Another reset would return FujiNet to normal/wifi mode with disabled BT. This is the function that disables the BT stack:

 

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/bluetooth/controller_vhci.html#_CPPv229esp_bt_controller_mem_release13esp_bt_mode_t

Trying to understand here. That would  fix it for no Bluetooth, but then what do you sacrifice when Bluetooth is active? Would there be any obvious 'memory cows' around to shut down to have enough RAM in this case?
 

Share this post


Link to post
Share on other sites
36 minutes ago, cathrynm said:

Trying to understand here. That would  fix it for no Bluetooth, but then what do you sacrifice when Bluetooth is active? Would there be any obvious 'memory cows' around to shut down to have enough RAM in this case?

In SIO2BT mode, the Fujinet is just a passthru device. The android app sends all the disk data over and Fujinet is effectively a disk drive only. So, all the other code for N, R, etc can be left uninitialized saving RAM.

Share this post


Link to post
Share on other sites
3 minutes ago, mozzwald said:

In SIO2BT mode, the Fujinet is just a passthru device. The android app sends all the disk data over and Fujinet is effectively a disk drive only. So, all the other code for N, R, etc can be left uninitialized saving RAM.

I see. Does the platform support DLLs or overlays or anything like this?  Or is it just compiled as one big blob?  Would it make sense to build two versions, an SIO2BT version and a regular version, with a bunch of #ifdef separating one from the other?   I've never used SIO2BT mode myself, do people who use this tend to switch back and forth?

Share this post


Link to post
Share on other sites
25 minutes ago, cathrynm said:

I see. Does the platform support DLLs or overlays or anything like this?  Or is it just compiled as one big blob?  Would it make sense to build two versions, an SIO2BT version and a regular version, with a bunch of #ifdef separating one from the other?   I've never used SIO2BT mode myself, do people who use this tend to switch back and forth?

Firmware is one big blob. I'm not really sure how or who uses SIO2BT mode. We could make two versions of the firmware, but if this idea works and can keep the functionality of both in 1 firmware it would be better.

Share this post


Link to post
Share on other sites
33 minutes ago, mozzwald said:

Firmware is one big blob. I'm not really sure how or who uses SIO2BT mode. We could make two versions of the firmware, but if this idea works and can keep the functionality of both in 1 firmware it would be better.

It sounds like the effort of this is to figure out what exactly can skip initialization if SIO2BT is still around, and then hard to know if that's enough to save SIO2BT. N:, R: do they really use that much RAM? I wouldn't think they need such big buffers, maybe something I don't know about here.

Share this post


Link to post
Share on other sites
Posted (edited)
5 hours ago, cathrynm said:

It sounds like the effort of this is to figure out what exactly can skip initialization if SIO2BT is still around, and then hard to know if that's enough to save SIO2BT. N:, R: do they really use that much RAM? I wouldn't think they need such big buffers, maybe something I don't know about here.

The ram that the N: and R: use, comes from the PSRAM (upper RAM, of which we have 4 megabytes currently mapped, and we can bank switch another 4 megabytes when needed), and we tried to go out of our way to make sure that malloc() and friends allocate from PSRAM first (sdk config). This is separate RAM from the 320k that's on the ESP32, and is connected via the SPI).

 

-Thom

Edited by tschak909

Share this post


Link to post
Share on other sites
1 minute ago, tschak909 said:

The ram that the N: and R: use, comes from the PSRAM (upper RAM, of which we have 4 megabytes currently mapped, and we can bank switch another 4 megabytes when needed), and we tried to go out of our way to make sure that malloc() and friends allocate from PSRAM first (sdk config). This is separate RAM from the 320k that's on the ESP32, and is connected via the SPI).

 

-Thom

I see, yeah, sounds like this is already optimized. I'm still stuck on the 'two versions' idea, I'm afraid. Would it be possible to build an SIO2BT version with the old tools, and the main version with the new tools? If so, then at least you could get unstick for now, and though the SIO2BT would be stuck on the old tools until you ripped out enough stuff to make it function,SIO2BT wouldn't be totally orphaned -- maybe it becomes a pain to build.  The main version could continue developing on new tools as before.

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...