freenode/#lisp - IRC Chatlog
Search
8:46:49
piotrbrzezinski
Yep. I just started (and have to learn emacs also on the way), but it feels great so far :)
8:59:03
phoe
Fun things: the CCL bug is an interaction between package-local nicknames, locks, and WITH-STANDARD-IO-SYNTAX in a multithreaded environment.
9:02:11
phoe
loke: CCL seems to be using cl:find-package, ccl::pkg-arg, and ccl::%find-pkg without much consistency as for which function to use at the moment
9:02:41
phoe
it's less of an issue with PLNs and more of an issue that CCL's code is messy like this
9:55:51
no-defun-allowed
in unrelated news, i got around to updating heroku-buildpack-cl to use new SBCLs and CCLs: https://github.com/nodefunallowed/heroku-buildpack-cl
9:56:41
no-defun-allowed
it was basically 4 LOC of shell to change, i figure the original author vanished?
11:06:13
dim
phoe: hey! did you send me an email to join the fun hacking and maintaining pgloader? ;-)
11:12:20
phoe
I have only basic postgresql experience and no experience with any other database systems, but I think I am somewhat competent with Lisp
11:25:31
no-defun-allowed
(even less relevant news: i found a typo in the CLHS! http://www.lispworks.com/documentation/HyperSpec/Body/m_w_smp_.htm is missing a - in "'something big")
11:34:32
_death
no-defun-allowed: do you know about https://www.cliki.net/ANSI%20Clarifications%20and%20Errata ? it contains this erratum in section 9. Conditions
11:50:56
phoe
dim: OK, I'll try to do some patches then. One of them is already present and waiting for PR and review.
11:51:31
phoe
I could also try fixing the travis builds and updating SBCL and CCL there, but I'd need some permissions for that. Where and how do I sign up for them?
11:51:52
dim
I have 11 unread issues at the moment and I'm not sure if I'll be able to have a look later today (late evening) or before
11:53:37
dim
thanks for the work! the manual parser for the SQL queries is a fun way to get started!
11:54:01
dim
I don't think it's exercized much or at all actually in the test suite, maybe in the geolite example
11:54:04
phoe
dim: I've noticed (: I'll need some help running regression tests for the parser - I hope you have those
11:54:49
phoe
a thing like that could use a series of unit tests and regression tests to check what gets parsed as what, along with all the edge cases that are invalid SQL and should be detected as such
11:54:51
dim
(pgloader:run-commands "pgloader/test/archive.load" :client-min-messages :debug) would be a good way to get started
11:55:50
dim
I'm not much of a unit tests person, more integration tests, and I have plenty of thoses... tbh the situation with testing in pgloader isn't as good as it should
11:57:49
phoe
this, and fix the CI as soon as I get the chance - the SBCL is so outdated it doesn't work with mainline ironclad anymore, and CCL fails my changes due to loading pgloader twice
11:57:55
dim
what I worry a lot about with unit tests is killing the ability to change your mind on internal APIs, which I've done a lot in pgloader
11:58:20
dim
I have several areas that are unfinished in terms of package/modules separation and generic function APIs and thigns
11:58:44
dim
for the parse-query case, I have zero excuse about not having proper unit testing of it, agreed
11:59:09
phoe
you're better off doing black-box integration testing on the APIs and ensuring that the final state is as expected
11:59:37
dim
it's been a one-man shop for a long time, everything might change at any point, even though on a daily basis they just don't
12:05:57
phoe
I see there are some means for people to buy pgloader subscriptions - are they being used?
12:32:39
dim
I mean I made the technical possiblity to help finance the project, but I never took the time to properly animate that channel and do Marketing around it, so there's no money for pgloader work, it's all on my free time
12:33:14
dim
now, that could change, I keep talking to people working in companies where they want more of pgloader, the problem is that I keep talking to engineers without a budget, not to their bosses
12:42:30
phoe
to me, it doesn't seem like raising financial support for pgloader would be too much of an issue - pgloader is a popular and demanded application, and it should be able to get some support contracts for it
12:42:56
phoe
and with that, actually hire some maintenance workforce that you (and they) can rely on
12:57:40
dim
I would like to have automated binary packages for instance, include continuous build system for windows and things, I just don't have time to make that happen
13:50:30
phoe
set a list of goals, e.g. hiring a second maintainer, fixing testing CI, introducing nightly pgloader builds for linux/win/macos
13:51:01
phoe
and throw the fundraiser at people who are interested and likely to throw money at pgloader
13:52:34
phoe
the general lispful population might also be interested, since pgloader is one of the most important/used/famous contemporary Lisp applications I can think of
13:53:42
phoe
if you need some help with setting that up, ask #common-lisp.net people - they have some experience with organizing such things
13:57:28
aerique
dim: These might be fun reads. Written by my 'boss' and you could also contact him on the technical people → people with money transition at possible customers:
13:57:28
aerique
• https://bert-hubert.blogspot.com/2015/07/developing-open-source-dont-listen-to.html
15:22:55
fivo
I pushed some path into asdf:*central-registry* and did an asdf:load-system then started with a new image again.
15:23:33
fivo
Now asdf still tries to load the system from that path even though asdf:*central-registry* only contains the quicklisp path.
16:06:15
dim
phoe: I also have a list of new features to add to pgloader, as per https://pgloader.io/roadmap/ ; one of the things I'd like to work on someday is an ABCL release were we would only use JDBC rather than specific database driver APIs as we do now, it would also help solving some issues with SSL and things
16:07:13
dim
I mean the ASDF dependency list would be different when using ABCL, because to be fair some of them are included in the JVM already, so we would just use them rather than load even pure-CL code (such as Postmodern), or some external libs (such as SQLite)
16:08:24
dim
the only reason why I avoided a fundraiser up to now is that I don't want people to invest their own personal money into this project, it's an enterprise thing, nobody would use that in their free time, I guess
19:18:27
defunkydrummer
@dim and decided to log in. I think PGLOADER is perhaps one of the hottest current tools written in CL right now, considering that Postgres is *THE* database.
19:19:13
defunkydrummer
dim: perhaps you should ask for people to collaborate on reddit/r/lisp. I didn't know you needed help with PGLOADER. If i had the spare time, I would.
19:24:22
defunkydrummer
dim: Question: Why does pgsql uses '(mssql sql) systems instead of using databases other than Postgres through CLSQL, which is already integrated with a lot of RDBMS including those two (and ODBC). (Of course, it doesn't have bulk copy operations...).
19:25:34
defunkydrummer
PuercoPop Oi Puercopop, entering IRC from an internet backdoor at *current-company*
19:29:00
defunkydrummer
@dim you wrote: "one of the things I'd like to work on someday is an ABCL release" -- however, it's likely that 95% of Pgloader users don't care (or know) what the underlying Lisp implementation is, and would rather simply download pgloader binary and run it. Mind you, i say this as a HUGE fan of the Right of Arming Bears and of Armed Bear Common L
19:30:13
defunkydrummer
sorry, my 2 cents. I'll keep more quiet now. Again, thank you so much for rewriting Pgloader, now with 100% less Python.
19:52:32
dim
defunkydrummer: I didn't like CLSQL when I had a look the first time, and wanted to have pure-CL drivers when possible, with a map like API to ensure that you don't have to load the whole SELECT result set in memory but instead can work in a streaming fashion
19:53:28
dim
defunkydrummer: ABCL would allow using JDBC, meaning pure-Java drivers for any and every source database system out there, from MS Access to IBM DB2 on OS/390; so that'd be quite a monivation for me and for some users too
19:53:46
dim
and other than that, thanks defunkydrummer for the kind words too, I feel humbled today!
19:54:38
defunkydrummer
@dim thank you for the tool. In fact here i'm steering the team towards moving to PostgreSQL, however we do have a strong need for SQL Server, and I use clsqlODBC
19:54:56
dim
fe[nl]ix: I tried to make it so that pgloader would be usable as a lib, but I'm not sure about it really, I don't have a use-case for that... well the command language parser produces lisp code, so... I kind of have a client
19:55:14
defunkydrummer
clsql+odbc to interface lisp with MSSQL . I didn't know pgloader could connect to mssql server. Today I Learned.
19:56:08
dim
in pgloader we use FreeTDS and a CFFI wrapper around it, it's not the best possible trade-off either, it kind of works usually, and FreeTDS looks well maintained
19:56:37
defunkydrummer
dim: yes, but i can access any and every source database system out there (pun intended). I don't remember what the cslql-odbc driver uses, but it works reliably, no complaints so far. What I need is bulk inserts.
19:57:10
dim
of course if a pure CL driver did exist I would use that, having to fiddle with .so or .dll things is not nice in CL when you ship a binary to end-users who know nothing about CL in general and setting up their .so loader in particular
19:58:54
dim
maybe I should have a look at clsql-odbc for perfs and memory usage characteristics someday, also I would accept contributions that would add support for such a driver in pgloader, maybe using a new source URI (e.g. odbc:mysql://user@host:port/db)
20:08:17
defunkydrummer
dim: having to use .dll files isn't so bad if you can legally bundle them with your system or if you know where to locate them beforehand.
20:11:51
dim
yeah well, debian and Postgres YUM packagers are doing a good job at that, other than that I know nothing about the system where pgloader is going to be invoked
20:12:17
dim
shipping a /usr/bin/pgloader application from a CL build system sometimes feels like being a pioneer, or maybe doing it wrong
20:27:35
defunkydrummer
i understand! Anyways, i need to log out and (invoke-restart #'perform-work)