HTTP+POST CELL_SIM7070 empty payload

[ipindado]

Hi...

I am having some issues integrating the PLUS B device with our backend. Our customer requires to integrate over HTTP with JSON payload as current plataform manage this protocolos, but I'm finding a strange behaviour when making some tests...

I am testing over telelogger, and after making some minor changes to adapt our backend I made it work (connectivity), but when I tried to integrate the telemetry it didn't work, while debugging I've tested with a dummy JSON content, which works correctly. The modem sets the body of the payload and sends the request.

Code: Select all

[AT REQUEST]: AT+SHBOD=25,100
[AT RESPONSE]: 1
[AT REQUEST]: {"a":1,"b":1,"c":1,"d":1}
[AT RESPONSE]: 1
[AT REQUEST]: AT+SHREQ="/request?deviceId=UCFLVA28",3
[AT RESPONSE]: 1

At the backend side I receive the request

Code: Select all

10.244.0.1 - - [12/Aug/2023:16:23:42 +0000] "POST /request?deviceId=UCFLVA28 HTTP/1.1" 200 12 "-" "curl/7.47.0" 207 0.010 [lab-application-lab-8080] [] 10.244.0.76:8080 12 0.011 200 5e6e6785702e7f290a86d869e4e4683c

And In the backend the content arrives correctly to our logs.

Code: Select all

----------------------
SENSOR REQUEST
date,request,assetConfig,configParams,scriptName,orgParams,files,services
inbox_http_enabled
headers,method,payload,sourceIPAddress,tenantId,parameters
POST
{"x-request-id":"5fa7258fb6a5691818485fb096b29658","content-length":"25","x-forwarded-proto":"http","x-forwarded-port":"80","accept":"*/*","x-forwarded-scheme":"http","x-scheme":"http","cache-control":"no-cache","user-agent":"curl/7.47.0"}
{"deviceId":"UCFLVA28"}
Payload: {"a":1,"b":1,"c":1,"d":1}

And here comes the strange behaviour... If I update the JSON content to add a new key to the object, the content arrives empty with 0 bytes to the the backend.

Note that it is the same dialog with the modem but with an additional key and increasing the body length and there is no error aparently

Code: Select all

[AT REQUEST]: AT+SHBOD=31,100
[AT RESPONSE]: 1
[AT REQUEST]: {"a":1,"b":1,"c":1,"d":1,"e":1}
[AT RESPONSE]: 1
[AT REQUEST]: AT+SHREQ="/request?deviceId=UCFLVA28",3
[AT RESPONSE]: 1

At the backend side I receive the request as previously (Note that size of the request is smaller 207-181=26 bytes ~ was the size of the first test body content, so it is seems it is same request but without any body)

Code: Select all

10.244.0.1 - - [12/Aug/2023:16:35:57 +0000] "POST /request?deviceId=UCFLVA28 HTTP/1.1" 200 12 "-" "curl/7.47.0" 181 0.012 [lab-application-lab-8080] [] 10.244.0.76:8080 12 0.012 200 db1bb1cde932c34e766517ed07eb91b4

And In the backend the content arrives without payload (0 bytes).

Code: Select all

16:40:10.913 [http-nio-8080-exec-5] INFO  e.a.s.s.a.i.r.CustomScriptEngineRestController:157 - Payload request with 0 bytes
16:40:10.913 [http-nio-8080-exec-5] INFO  e.a.s.s.a.i.r.CustomScriptEngineRestController:287 - String media type: text-plain-null
16:40:10.913 [http-nio-8080-exec-5] INFO  e.a.s.s.a.i.r.CustomScriptEngineRestController:289 - text_plain content
----------------------
SENSOR REQUEST
date,request,assetConfig,configParams,scriptName,orgParams,files,services
inbox_http_enabled
headers,method,payload,sourceIPAddress,tenantId,parameters
POST
{"x-request-id":"56a8b18318e94c15686b7d008d00ffa4","content-length":"0","x-forwarded-proto":"http","x-forwarded-port":"80","accept":"*/*","x-forwarded-scheme":"http","x-scheme":"http","cache-control":"no-cache","user-agent":"curl/7.47.0"}
{"deviceId":"UCFLVA28"}
Payload: 
Payload: ""

Default BODYLEN request is setup to 1024 in the FreematicsNetwork.cpp, so it should be enough to handle just 6 more bytes, but I've tried also to set it to 4096 (Max value for SIM7070 according to documentation)

I've checked for additional settings of the modem in the documentation but I don't see any relevant...

Anybody has an idea of what can be the problem?

Best recargs

[stanley]

Try increasing RECV_BUF_SIZE in FreematicsNetwork.h.

[ipindado]

Thanks for the response...

I tried out increasing RECV_BUF_SIZE but it didn't improved. After many trial and error I finally figured out...

Problems seems to be in FreematicsNetwork.cpp:804 in the AT+SHBOD command initially sent to the modem initialize the request body..

Code: Select all

      sprintf(m_buffer, "AT+SHBOD=%u,100\r", payloadSize);
      if (sendCommand(m_buffer, 100, "\r\n>")) {
        sendCommand(payload);
      }

The AT+SHBOD command is sent to the modem with a timeout of only 100ms, and the sendCommand itself adds a delay of 50ms after writing the command to the modem through the UART port(FreematicNetwork.cpp:540)... So, it seems that it just leaves 50ms to complete the set body command. When the body is small it works fine, when it is a little bit longer, it works randomly and when it is larger it never works...
If I understood correclty in the esp-idf docs, the driver is configured to block the transmision to the COM port until there is enough space in the FIFO, so this time may be different each time.

Finally just increasing the timeout of the AT+SHBOD command, made everything work correclty...

Code: Select all

      sprintf(m_buffer, "AT+SHBOD=%u,1000\r", payloadSize);
      if (sendCommand(m_buffer, 100, "\r\n>")) {
        sendCommand(payload);
      }    

I tested both with HTTP and HTTPS and seems to work perfectly with the max body size of the modem (4096) all the times. I don't need this body size, but it is good to know that it works.

Code: Select all

10.244.0.1 - - [15/Aug/2023:15:12:38 +0000] "POST /request?deviceId=UCFLVA28 HTTP/1.1" 200 12 "-" "curl/7.47.0" 4280 0.189 [lab-application-lab-8080] [] 10.244.0.82:8080 12 0.023 200 ebd304335d95d038bc482f8009dcd646
10.244.0.1 - - [15/Aug/2023:15:13:00 +0000] "POST /request?deviceId=UCFLVA28 HTTP/1.1" 200 12 "-" "curl/7.47.0" 4280 0.193 [lab-application-lab-8080] [] 10.244.0.82:8080 12 0.023 200 48d6f2211bdd4a8c65c2106a7f5ce2e1
10.244.0.1 - - [15/Aug/2023:15:13:22 +0000] "POST /request?deviceId=UCFLVA28 HTTP/1.1" 200 12 "-" "curl/7.47.0" 4280 0.188 [lab-application-lab-8080] [] 10.244.0.82:8080 12 0.023 200 0e5e54023394c3f1dc83476ab9ffb844
10.244.0.1 - - [15/Aug/2023:15:13:58 +0000] "POST /request?deviceId=UCFLVA28 HTTP/1.1" 200 12 "-" "curl/7.47.0" 4280 0.197 [lab-application-lab-8080] [] 10.244.0.82:8080 12 0.022 200 e6cb9d30df65047f34be0ff9372cbecc

[stanley]

Great. I will fix the code in github according to your finding. Thanks.