freenode/lisp - IRC Chatlog
Search
8:03:52
Nilby
There's this 10̦000̦00 or this 10̲000̲00 but I think they're more trouble than they're worth.
8:05:45
fiddlerwoaroof
moon-child: copy-pasting isn't a huge concern, but I find it irritating when I have to reformat basic data types between langauges
8:06:39
fiddlerwoaroof
Anyways: Excel and other spreadsheet tools have all sorts of nice number formatting options
8:06:58
fiddlerwoaroof
It'd be great for me to be able to set such display options on a variable-by-variable basis :)
8:08:02
White_Flame
hmm, but when copying a cell (at least in libreoffice), you get the presentation form, not the raw form
8:10:22
fiddlerwoaroof
And, I don't think Excel has that limitation: at least, you can choose whether to paste the display formatting or the values at paste time
8:12:13
White_Flame
yep. The general problem IMO is the Unix/C model of naked char buffers for everything
8:13:47
fiddlerwoaroof
In fact, macOS's biggest technical advantage is that it's _not_ based on the Unix/C model of text everywhere for the user-facing parts
8:14:16
fiddlerwoaroof
It's based on the Objective-C runtime, which is basically a limited Smalltalk
8:20:13
splittist
The most important syntax is the one I'm presenting to myself in my source code: did I just type one hundred million or a billion? Let me move my cursor over the number as I count... 100_000_000 makes it obvious to my most important reader - me (:
8:23:33
jdz
Usually by moving cursor to position 3 from the right, and then selecting 3 next digits left.
8:25:56
jdz
#r should be extended to accept #\" as the base, so then: #r"one hundred million billion".
8:26:28
fiddlerwoaroof
this is sort of interesting, in Emacs: (gui-get-selection 'CLIPBOARD (intern "text/html"))
8:26:58
fiddlerwoaroof
(gui-backend-get-selection 'CLIPBOARD 'TARGETS) shows all the possible target formats
8:29:57
Nilby
The natural way of representing numbers is of course recursive, so any linearization into bits is somewhat fake.
9:04:31
contrapunctus
Shinmera: it seems you changed the licenses for some of your projects from Artistic to zlib, but the accompanying web pages still name the former as the license. I figured you might want to update them, to avoid confusion.
9:13:50
contrapunctus
Shinmera: is your website not in a repo I can make a PR for? 🤔 I couldn't find a repo. Or, wait, are those generated docs?
9:15:01
Shinmera
each repo has a docs/ folder. That's where the sources are for those. And yes they're generated, but once per repo, so it's a pain in the ass.
10:51:28
Demosthenex
ok, i'm trying to work with a script i wrote that import's a library using it's pathname, but that seems to fail in sly/slime
10:56:33
Nilby
You could use #+(or swank slynk) . I'm not sure what you mean by import though. It's probably best to make it work without special code.
10:59:35
Demosthenex
Nilby: i'm trying to make it so that i can eval my buffer to load it into repl, and there are assumptions made about the path... so i'd like to wrap that in some code wwhich detects that i'm running in swank/sly/slime so i can address the mising data
11:07:40
Nilby
Ah. There's lots of ways to address the issue. #+ conditionals are the simplest, but maybe most brittle. You could also use logical-pathname-translations, to make a path prefix, then say (load "my-stuff:foo.lisp") But, loading with asdf or quicklisp is probably the most usual.
11:18:02
Demosthenex
maybe i need to wrap my final code in a main function, and then only call that when i use --script
11:40:41
Demosthenex
is there a way i can say "call (main) only if called from sbcl --script, ie: not sly?"
11:45:26
Nilby
But perhaps it's best to put whatever you want to do only interactively in separate function.
13:46:18
rumbler31
Xach: I don't know if this would be useful to have in quicklisp, but I wrote a function to copy a single local project into the bundle directory. Problems that I see with adopting it are that I don't yet account for dependencies that might need to be pulled in from other local projects, and discovery of the bundled location on the user's behalf (which might be impossible)
13:46:34
rumbler31
the goal is to reduce the amount of code that makes it into the bundle, I don't want all of my local projects to get pulled in
13:51:37
rumbler31
I'm not in front of the code but I think I use asdf to find the named package locations, make pathnames for each of the .lisp and .asd files found in that directory to the location of the bundle/local-projects/myproject folder and does a uiop file copy
13:54:47
Xach
rumbler31: I'd look at what ql-minitar does, I think, except copying from filesystem files rather than a tar stream.
13:56:26
brandflake11
Hello all. Have you worked with extremely large datasets in lisp before? I'm trying to load in a huge list to use with lisp (800000+ lines), but it seems like sbcl freezes everytime I do that. Do I just have to wait for it to load in, even though sbcl says it's not taking any cpu? What has been your experience with large datasets in lisp?
13:56:56
Xach
brandflake11: sometimes the freeze is when it tries to print the result, not loading or working with it. one option is to set *print-length* to something like 100.
13:57:21
Xach
brandflake11: using too much memory results in a different symptom (landing in ldb or crashing outright)
13:58:48
Nilby
I've used much bigger datasets and the important thing is set --dynamic-space-size high enough, and don't print it.
13:59:24
brandflake11
Nilby: I don't know anything about dynamic-space-size. Do you set this when loading sbcl, or can you set it with a global variable?
14:01:26
phoe
CLHS says, "If [*PRINT-LENGTH*] is false, there is no limit to the number of components printed."
14:01:53
brandflake11
Oh, I didn't know the hyperspec had info about global variables that are important.
14:07:18
brandflake11
Nilby: I took a .csv file and converted it into one big list of lists to then parse out later as OSC commands. At this point in my lisp journey, it seemed like the simplest option.
14:08:03
brandflake11
But I'm still having trouble getting the list to even load. I need to try to dynamic-space-size option
14:08:46
phoe
brandflake11: how does it hang? if you're using slime/sly, check the inferior-lisp buffer
14:08:59
phoe
maybe you need more heap to be able to load this much at once, if you get an error about heap exhausted in there
14:09:35
beach
brandflake11: 800000 lines doesn't sound like much, depending on what a "line" is of course.
14:10:48
brandflake11
phoe: You are so right, I get this "Heap exhausted during garbage collection: 48 bytes available, 64 requested. "
14:11:26
jdz
Also it might be a good idea to process lines as they are parsed from the file (say by providing a callback function to run on each parsed line) instead of doing it in two stages.
14:13:11
brandflake11
jdz: Well, the list is a dataset with times of events and the events themselves. So, I want to be able to replay these events in order
14:17:07
jdz
I use a couple tricks when doing this on SBCL: a) if the text does not have any unicode characters coercing each line into SIMPLE-BASE-STRING, and b) if there are fields that may have duplicate (string) values longer than 8 characters then using a hash-table to de-duplicate them.
14:17:51
brandflake11
When doing this option with slime, do you just set the option in emacs's inferior-lisp-program variable?
14:18:53
phoe
brandflake11: my elisp variable, inferior-lisp-program, is "ros dynamic-space-size=8192 -Q run"
14:19:59
Nilby
I just tested reading a 800+k line 217.3M csv with the naive list reader with --dynamic-space-size 4096 and it was fine. (The slow part is guessing the column data types)
14:20:43
brandflake11
I just set it to "/usr/bin/sbcl --dynamic-space-size 2048". I'll see what happens
14:22:13
brandflake11
lol, I have a fast internet connection, downloading more ram should be no problem XD
14:25:20
brandflake11
I get DYNAMIC-SPACE-SIZE is unbound, so I'm guessing setting it where I did with the emacs variable didn't work
14:28:28
brandflake11
Oh man, I'm glad you all helped me with this. I would not have been able to figure this out on my own
14:30:23
brandflake11
I was really worried lisp just wasn't able to handle huge datasets, but i'm glad that it was just a matter of increasing the ram allowed to it
14:30:56
beach
brandflake11: Either way, it would not be a language limitation, but a limitation on specific implementations.
14:32:03
beach
brandflake11: Also, it is not RAM that you are assigning, it is just heap size. If you have virtual memory, that would work too, but more slowly.
14:32:28
Nilby
There is even hope on the horizon that sbcl will be able to increase it's own memory.
14:35:46
beach
ldbeth: Probably not. You would somehow have to teach the garbage collector where to find it. Plus, the way a lisp object is represented is not standardized.
14:37:39
brandflake11
I just got the database list assigned as a variable. You guys all rock. Thank you so much for the help and teaching me something
14:39:37
jcowan
back to numeric representations: in Scheme, #e forces a float-style syntax to be implemented as an exact number: thus #e1e20 is the integer 1^20, and #e1.5 is 3/2.
14:40:50
jcowan
Note that this is not just a coercion: 1e400 is normally infinity, but #e1e400 is the integer 10^400.
14:41:14
phoe
I don't think there's a standard CL way other than #.(* 1 (expt 1 20)) for integers and #.(rational ...) for ratios
14:44:11
brandflake11
Nilby: maybe I did something wrong. I got it all loaded in though, so that's what I'm happy about
14:46:42
brandflake11
Maybe I should just use cl-csv instead now. I couldn't get that to load in the .csv file (I didn't know about checking *inferior-lisp* buffer to see that the heap was too low) so I turned the .csv into a list of lists using bash sed.
14:48:55
brandflake11
("0.000000" "1" "68" "92.45.54.178" "10.50.209.134" "" "" "" "" "" "116" "UDP")
14:51:22
beach
For example, if you have lots of empty strings, then the reader will allocate a separate empty string for each one. That could become quite wasteful.
14:53:10
beach
You could do that, but if you do it after the thing is read, then you still need as much space, at least temporarily.
14:57:46
beach
brandflake11: When I asked about duplication, I didn't necessarily mean the full list in your example, but also individual elements such as "1" or "10.50.209.134".
14:59:17
brandflake11
beach: I'll look into hash tables. I have never used a hash table before. It's not hard to grasp is it?
15:00:15
beach
brandflake11: Does your file consist of a single huge list, where each element is a list as in the example above?
15:01:35
brandflake11
beach: It's pretty much my first serious programming language. I am from the Music and Composition and Linux world, where I learned pure data as my first language. I learned a little bit of C++ on my own time last year, and then discovered common music, which brought me into common lisp. I love the syntax and realtime feeling of common lisp, so I've been learning it now to do other stuff
15:02:35
phoe
wrt books, you might want to read either Gentle or Practical Common Lisp; I'd recommend that you start with the former, and if you find it too gentle indeed, then switch to PCL
15:02:42
minion
brandflake11: look at gentle: "Common Lisp: A Gentle Introduction to Symbolic Computation" is a smoother introduction to lisp programming. http://www.cs.cmu.edu/~dst/LispBook/
15:02:46
minion
brandflake11: please look at pcl: pcl-book: "Practical Common Lisp", an introduction to Common Lisp by Peter Seibel, available at http://www.gigamonkeys.com/book/ and in dead-tree form from Apress (as of 11 April 2005).
15:03:12
brandflake11
phoe: Oh yeah, I've gotten to chapter 3 of practical common lisp (not very far I know). I need to keep working at it
15:03:38
brandflake11
Actually people on here recommend I read Practical Common Lisp. It was all thanks to this IRC channel!
15:03:39
beach
brandflake11: Then, what I would do would be to not read the entire thing with a single READ, but to read one element list (as in your example) at a time, then process it by checking whether an element is in a hash table, and if so, reuse it.
15:04:45
brandflake11
beach: Thank you I really appreciate your suggestion. I'll look into doing that to reduce memory
15:04:48
phoe
in my opinion, PCL chapter 3 goes straight into business without explaining that the first practical chapter with disc DB is likely not meant to be fully understood straight away and is rather meant to give the reader an overall feeling of how to work with the language
15:05:14
phoe
it's the only explanation I see for "chapter 3 is where we make a database, and chapter 4 is where we actually start explaining what we've done"
15:05:44
brandflake11
phoe: I stopped reading it because I was more interested in making music with Common Lisp. I've been reading Heinrich Taube's "Notes from the MetaLevel" at the moment instead to get more music making done
15:06:15
brandflake11
phoe: Thank you though, I plan on keep going with it. I want to be able to do cool stuff with lisp
15:08:06
brandflake11
phoe: There is a version of Common Music that integrates a realtime midi out using incudine. That's the version I'm using now
15:08:47
brandflake11
I'm able to use emacs as my music composition tool, which is really cool and spit out midi through jack to my synths and other toosl
15:50:21
kevingal
I've had this in my bookmarks for ages, been meaning to try it out: https://nunotrocado.com/software/cl-collider-tutorial-1.html
17:55:29
CL-ASHOK
How do I "minimise" emacs terminal? I'm trying to run a Hunchentoot process, but want to return to the terminal (this is on a VM)
17:58:04
CL-ASHOK
@xach: what is that? (I'm currently remote into a Google VM with the non-gui emacs)
18:04:01
Bike
it's not as sophisticated, but you could also suspend the emacs process with probably control z
19:56:42
nij
Hello, is there any functional "package manager" (or rather, system manager) for common lisp? I'm thinking of it as an analogue of straight.el (for elisp).
19:58:35
Shinmera
nij: what does it mean to 'reproduce all systems'? Download them? Load them into the image?
19:59:03
nij
For example, straight.el lets you do this : (el-patch :type git :host github :repo "raxod502/el-patch")
19:59:30
Shinmera
just put (ql:quickload '(a b c)) into a file then or is that not declarative enough
19:59:41
nij
With this in a declarative file, which is run by the package/system manager, the corresponding package/system will be fetched.
20:01:05
Shinmera
No, quicklisp packages snappshots of the ecosystem so that all the libraries are versioned as one thing at once, since usually systems do not specify versioned dependencies.
20:01:06
nij
E.g. I might want ql to fetch the system from a specific repo (maybe even local!).. and I might want a specific version/commit.
20:02:22
Shinmera
ASDF cannot deal with knowing multiple versions of the same system at once. It'll only accept one.
20:02:22
waleee-cl
nij: if you put it in ~/common-lisp or the appropriate subdir of ~/quicklisp it will be preferred
20:02:42
Shinmera
So you'll have to either manage the system registry manually, or replace ASDF wholesale.