[elm-discuss] Decoding json from incoming websocket packets

[_Boris _]

Hello,

I have implemented very straightforward converting received packet into Msg:

subscriptions : Model -> Sub Msg
subscriptions model =
Sub.batch
[ WebSocket.listen wsServer decoderServerMsg

where decoderServerMsg:
decoderServerMsg : String -> Msg
decoderServerMsg rcvdStr = case JD.decodeString jsonDecServerMsg rcvdStr of
Ok serverMsg -> ServerMsgReceived serverMsg
Err err -> SystemError ("Failed to parser server msg:"++err++"
string:"++rcvdStr)


It works well as long as there is no fragmentation, ie. every packet
contains valid json payload.
My problem starts when packet exceeds about 4500 bytes and then it is
broken into two chunks.
I believe it very common case so before reinventing the wheel I wanted to
ask if there is already solution for this problem that I can use?


Thanks,
Boris

[Alex Barry]

If you control the sender and are guaranteed to get some messages exceeding
the websocket limit (which I'm assuming is 4500 bytes) is to either send
multiple smaller updates, or assign some sort of message id and prepend it
to each chunk.

Another option you can do (if you can't control the sender) is have a
string buffer, and each time you get data, if it can't be decoded, append
it to the buffer. After every append, attempt a decode, and if it's
successful, clear the buffer. This will probably break if you get multiple
unrelated messages.

One thing you should keep in mind is websockets on their own aren't really
for messages, it's for streaming data. If you can guarantee synchronous
messages (ie first message must always complete before the second message
starts), you could send a byte length + json (ie "14{ foo: 'bar' }", and do
the buffer method I suggested.

So if you have big messages, your best bet is to chunk them yourself so
you're guaranteeing the size. Otherwise, hope they are synchronous and
assemble them in elm.

Post by _Boris _
Hello,
subscriptions : Model -> Sub Msg
subscriptions model =
Sub.batch
[ WebSocket.listen wsServer decoderServerMsg
decoderServerMsg : String -> Msg
decoderServerMsg rcvdStr = case JD.decodeString jsonDecServerMsg rcvdStr of
Ok serverMsg -> ServerMsgReceived serverMsg
Err err -> SystemError ("Failed to parser server msg:"++err++"
string:"++rcvdStr)
It works well as long as there is no fragmentation, ie. every packet
contains valid json payload.
My problem starts when packet exceeds about 4500 bytes and then it is
broken into two chunks.
I believe it very common case so before reinventing the wheel I wanted to
ask if there is already solution for this problem that I can use?
Thanks,
Boris

[_Boris _]

Thanks Alex, your response makes perfect sense. It definitely makes sense
to treat data received on websocket as stream as opposite to discrete
messages. Some challenge here is absence of indication when connection got
closed and reopen (to reset the buffer). Maybe need to consider using
elm-lang/websocket.
Also it is a bit surprising that such common task does not have ready-to-go
solution. Alex, thanks again for your response.