[elm-discuss] Json file size

[David Legard]

I have been porting a language translation engine from HTML/JS to Elm, and
it was easy to implement.

The problem is that the dictionary contains 15,000 word pairs.

I can't include those items as Elm files (since Elm-make runs out of
memory), so I decided to put them in a JSON file and import that.

It works when I load the application, but as soon as I start interacting
with the app, the console logs the error:

Main.elm:1381 Uncaught RangeError: Maximum call stack size exceeded
at Object.cmp (Main.elm:1381)
at Function.compare [as func] (Main.elm:1162)
at A2 (Main.elm:92)
at Function.func (Main.elm:4598)
at A2 (Main.elm:92)
at Main.elm:8952
at Main.elm:16
at A2 (Main.elm:93)
at Function.func (Main.elm:5075)
at A2 (Main.elm:92)

Does anyone have any idea how I can circumvent this problem? Thanks.

[Peter Damoc]

Use the dictionary as a JS service.

You can load the dictionary in JS and use ports to interrogate it.
https://ellie-app.com/3Z8hSkXxxBGa1/0

If you like to live dangerously you can also use Kernel code to interrogate
the JS dictionary datastructure synchronously.
Since the dictionary is fixed, the purity of the function is maintained and
this falls into one of those extremely rare use cases where one drops to JS
for performance reasons.

Post by David Legard
I have been porting a language translation engine from HTML/JS to Elm, and
it was easy to implement.
The problem is that the dictionary contains 15,000 word pairs.
I can't include those items as Elm files (since Elm-make runs out of
memory), so I decided to put them in a JSON file and import that.
It works when I load the application, but as soon as I start interacting
Main.elm:1381 Uncaught RangeError: Maximum call stack size exceeded
at Object.cmp (Main.elm:1381)
at Function.compare [as func] (Main.elm:1162)
at A2 (Main.elm:92)
at Function.func (Main.elm:4598)
at A2 (Main.elm:92)
at Main.elm:8952
at Main.elm:16
at A2 (Main.elm:93)
at Function.func (Main.elm:5075)
at A2 (Main.elm:92)
Does anyone have any idea how I can circumvent this problem? Thanks.
--
You received this message because you are subscribed to the Google Groups
"Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

[David Legard]

Thanks, I'll look through that code.

A couple of minor points -- I am not using a Dict, simply a list of records
holding the word pairs. Is Dict better performing in this situation?

My use case is slightly different - as the source language (Thai) is
written without spaces between words, I need access to all the Dict keys
when parsing a text to determine where the word breaks go, so it's not just
a matter of interrogating the database.

Also, the Dict items need to be sorted in a particular way (by length of
the key), which is an added challenge.

Anyway, thanks for showing me a new direction to pursue.

[Ian Mackenzie]

I suspect there will be a way of making this work without dropping into JS. It looks like you're hitting some sort of very deep recursion which is causing a stack overflow - try seeing if you can isolate which of your function calls is triggering the stack overflow, and see if you can change some function from recursive to non-recursive (or tail recursive). If you have example code, that would also be helpful...

[Steve Schafer]

I think your problem is this bug:
https://github.com/elm-lang/elm-compiler/issues/1521

The workaround is to disable the Elm debugger.

Post by David Legard
Thanks, I'll look through that code.
A couple of minor points -- I am not using a Dict, simply a list of
records holding the word pairs. Is Dict better performing in this situation?
My use case is slightly different - as the source language (Thai) is
written without spaces between words, I need access to all the Dict keys
when parsing a text to determine where the word breaks go, so it's not just
a matter of interrogating the database.
Also, the Dict items need to be sorted in a particular way (by length of
the key), which is an added challenge.
Anyway, thanks for showing me a new direction to pursue.

[David Legard]

I ended up going the JS/ports route and it works fine now, so thanks for
the suggestion.

It was quite a shock going back to the catalogue of atrocities that is
Javascript after so long writing in Elm. It was like being thrown back into
a medieval time of druids and burning witches. Twenty lines of Javascript
caused me more pain than 700 lines of Elm.

One thing I came across which I haven't seen documented is the way to write
Unicode literals in Elm (I have to work with the Thai alphabet, which is at
\U0E00 on up.)

Anyway, Elm likes its Unicode literals as

gorgai : Char
gorgai = '\x0E01'

[David Legard]

The other 2 modes of doing this failed as follows:

1) db.json. The JSON loaded fine, according to the Debug.log, but the
program failed on the first user input.

That is, when I changed the input

[textarea [cols 70,rows 20,placeholder "Enter text here", onInput Change ]
[]

...

Change s -> ({model | sourcetext = s},Cmd.none)


.. the call stack error occurred. Theoretically, the Decoding had occurred
prior to any user input, but who knows


2) The pure Elm approach

I began with the dictionary in one huge Elm variable

List (String,String)


then, after doing some reading about compiler crashes, cut it into several
files.


dictionary : List Word
dictionary = [D1.words,D2.words,D3.words ..... ] |> List.concat |> makeWords

makeWord : (String,String) -> Word
makeWord (e,t) = Word e t

makeWords : List (String,String) -> List Word
makeWords = List.map makeWord

But it still wouldn't compile.