Wiki/Report of Meeting 2023-11-02

Report of Meeting 2023-11-02

Present: Ed Gottsman, Raul Miller, and Bob Therriault

Full transcripts of this meeting are now available on this wiki page.

1) Ed plans to record a screencast of the demo and send the files to Bob so he can edit and post it up on YouTube. Ed feels the GitHub search tool will be useful for intermediate users because it provides some options in ways to use the code. Bob thinks that the important thing is that the code that is found is actually being used and not just the way that J coding is prescribed. This may be a very good way to onboard a J programmer to common coding practices. The GitHub search might also be a way to see changes that had taken place as new primitives are introduced. Ed plans to introduce the GitHub tool at the next meeting of the British APL Association meeting.

2) Raul raised the point that there is a difference between the Dictionary and NuVoc and that not all the information in the Dictionary has made its way across to Nuvoc. Bob wondered whether NuVoc might be administrator only edited to maintain the veracity of the source. Then there was a discussion on labs and the ways that these might be incorporated into use for the wiki and NuVoc. J Playground could accomplish this by loading labs into the edit window and then could be accessed from the terminal line by line.

3) Bob asked about the way that breadcrumbs were used in the wiki. In the new category tree, Bob has removed the breadcrumbs because they are redundant. If a link on the category page takes you to a new page that is not a category page then it might have a breadcrumb. Ed pointed out that there is a back arrow on the browser that will always take you back. Ed pointed out that two steps away would still be disconnected. Bob said that the sidebar would still have the main categories and there would be access to the map of the site on every page under the search bar.

4) Bob thinks that the J wiki search tool is working very well and feels that it works well with the categorization. Ed felt that it is really important to use the categorization because that is accessible for anyone with a browser. The J wiki search is only available through JQt. Ed points out that curation makes the ball of information in the wiki more accessible, but what he has done is more about presentation and the J wiki browser is the thing that accentuates the curation through good presentation.

For access to previous meeting reports https://code.jsoftware.com/wiki/Wiki_Development

If you would like to participate in the development of the J wiki please contact us on the general forum and we will get you an invitation to the next J wiki meeting held on Thursdays at 23:00 (UTC) Next meeting is November 16, 2023.

Transcript

So I hope to have background download working in the next couple of days and then I'll try to do a demo video and send it to you for editing.

OK.

But that's where I stand now, but I think we're pretty close.

I dumped you into the 90-90 rule, huh?

So the 90-90 rule, what is that?

You do 90% of the work for 90% of the benefit?

Yeah, the first part, first 90% of the work takes 90% of the time and the remaining of the work takes the other 90% of the time.

Yes, exactly, exactly.

And this time the culprit seems to be Windows or my lack of understanding of the Windows file system.

I'm making progress, but I'm not quite there yet, I think.

It'll be OK.

OK, well, if you want to do your local recording and then just fire off the files, and that should allow you to do it, you know, fairly piecemeal.

Like, don't feel like you've got to do the full thing in one roll because I can put it together.

Well, that's very kind of you.

I know that's work, so I very much appreciate it.

Yeah, it's not as much as you think.

If you do it that way, the one thing I would like is some screenshots.

And that's because if I have to splice your voice together and I've got a screenshot to cover it, piece of cake.

I see.

And basically screenshot means, well, it'll all be screenshot, but that just means silence.

Is that what you're saying?

Yeah, if you want to take a retake, just take a breath and then start again.

And then I can easily cut on those.

That's not a problem.

OK, excellent.

You're very kind.

Thanks.

Yeah.

And, you know, if you want to make it really easy for me, after you've put your video together, just do a quick log about what takes you want to use.

And I can immediately go to those and pull them out so I don't have to be guessing at what you like best.

Right.

Excellent.

Thanks.

I may or may not agree with you.

Well, you're the editor.

You get the final cut.

That is exactly right.

But I'm a nice editor, so I think you should have respect.

I'm sure that you are.

I've met editors who weren't so nice and got great results, but people didn't want to work with them.

Exactly.

And then I still think that the GitHub search that shows you the most common following characters for the code sequence that you're looking at, I still think that's interesting.

And I think if it's interesting at all, Raoul's right, it's interesting for intermediate learners, not for beginners, not for experts who wouldn't need it.

But the whole question of what are the idioms, what are the locutions that are common in usage of the language.

Sometimes NUVOC answers that, maybe.

But a corpus, you know, access visibility into a large corpus tells you a lot more than NUVOC is going to be able to tell you about how a primitive or a sequence of primitives is actually used in practice.

And I think that's important.

That's what I want to actually talk to the British APL people about, is how could that be enhanced?

Are there other things we can do now that we've got our arms around a code corpus?

Are there other kinds of instructional or reference mechanisms we could put into place?

I don't know if it's a sensible question to ask, but I'm going to pose it.

I think it's a really sensible question to ask.

I think its use is, like you were asking about where that might fit into NUVOC or where that information might be in NUVOC.

I think the place I've seen it in NUVOC is in the examples.

Yeah, for sure.

People will show an example, but the examples won't be as comprehensive of what you're seeing in code.

And I think another key thing is the examples will be somebody's interpretation about how something should be done.

Whereas what you're doing is you're showing what people are doing.

And I think that is actually the most important part.

You're no longer saying, we should do it this way.

You're saying, this is how it's being done.

And that tells you actually a lot more for somebody who's trying to develop in the language.

I think it's almost called positive, or maybe the philosophers, I don't remember, positive rather than normative.

In other words, what actually is happening versus what ought to under the norms.

So yeah, that's an interesting way of looking at it.

That's true.

And I'm starting to think about how NUVOC is trying to, you know, originally we had the dictionary, we had labs, and then we kind of messed around with the labs a bit and started letting them pull out a favor.

And so I think NUVOC is trying to fill in for some of the lab content in the dictionary, in a dictionary like format.

I don't know how well that's working.

Yeah, I think.

Yeah, and I think, I think NUVOC originally when it was started was meant to be just what you're saying, sort of a more expansive dictionary that wasn't as terse, but was maybe a bit more accessible to people who were starting out.

And I think what it evolved to, because it was changeable, then the decision was made to make it the actual reference rather than the dictionary, like the hard dictionary.

Now, you had something that could flex to fit what the language was actually doing.

It was a living dictionary.

But I think that puts it in a place it actually hasn't meant to be, but it's inherited that, and I think it's still trying to find its legs to be able to do that.

Yeah, I remember, you know, Ken Iverson quip about how one of the primary skills of a language designer is the ability to say no, and wikis don't have that.

And NUVOC as a authoritative reference is struggling with that, I think.

Yeah, I think, well, I think there are things, and I guess it involves the wiki because NUVOC's part of the wiki, I would guess if you wanted to make it more authoritative, you'd take away privileges other than maybe administrators or some other higher level to be able to change it.

Yeah, and then you'd have to have a plan for how to clean it up too.

Yeah, but if you did that, then you should have a plan for how to clean it up because then there's a level of commitment.

You've given people a level to be actually to be able to change it, and the commitment would be in exchange for that ability to make changes, the expectation is we will make it better.

I think that's true as far as it goes, but there's still the whole lab side where the content that's in there is useful.

And it needs, I mean, from my perspective, it needs use case, it needs to be, you know, instead of clustering around the reference work, clustering around use cases, and the work needs to be put in to build out those use cases, build out the labs or whatever else, the demos that it's got so that we're not losing that, we're just moving it to a more suitable place.

Well, out of my walk, I mean, it's wild that you should mention that because out of my walk, it occurred to me, and I haven't tried this yet, but you can trigger an instance of the playground with loading up a source file and put that in a link.

We know we can do that.

What I'm not sure about, but I think we should be able to do is you should be able to load a lab the same way.

Like the way Eric's done the labs, they're just text files.

And if you load a text file in, and again, I haven't tried it, but there should be a way that a text file, when you load it in, then you load that lab in, it should respond the same way through the playground as the existing labs do.

Now, I don't know whether that's true, because I'm not sure of the mechanism of the existing labs that come off the menu.

But if that were true, then you just have to develop a lab for each primitive in NuVoc that gives you examples and different things.

I see what you're saying.

So not a lab in the classic case where there's some practical thing that we're trying to teach, but more of a demo situation.

Kind of like a primer than a lab, except for it's an interactive primer.

I did some video labs for different primitives, like head and tail and various things.

And so there's actually, I think, links in NuVoc to them.

They've never taken off because nobody, I think, wants to sit through a video.

And the video is not, well, I broke it into like, I think it was 20 or 30 second segments, and then there would be a lab information after that.

So it made it a little more accessible.

But I'm thinking really what you want is just the same thing.

But don't worry about doing a video, just do a lab and it shows you the different, like, for instance, the difference.

Here's the thing I'm thinking is, if you have a lab on a primitive, you already have somebody who wants to understand that primitive.

You're talking to maybe a beginner who's curious or a classroom situation.

That requires some lead up that, you know, something to generate interest in the first place, you know, to get them there.

Whereas the labs in the original sense were covered a topic and brought in Jay as a way of getting you to that topic, which is a much more interesting.

A person tends to burn out on the simple stuff, you know, it's like, this is so simple.

Why can't I remember what's going on here type of thing when you have too much focus on just the word, just the semantic, just the symbols.

You got to have other things to engage your interest to keep your mind happy.

That's what I'm thinking right now.

So applications rather than primitives.

And I've been trying to run use cases because that's a fairly common term in the programming community for that perspective where it's more than just, it's not quite a finished product.

It's not quite the language, but it's something that a person can sink their teeth into.

Where are the labs stored?

In Jay?

I think they're under add-ons.

They're under add-ons.

No, I meant like repository.

Are they in GitHub somewhere?

Yes.

Now, the interesting thing about labs is most of them are in the GitHub repository under labs, but they don't have to be.

You can put labs anywhere if you identify them, and then put a manifest that will direct you to the right part of it.

So not every lab, like I think, oh, Cliff Writers, one of Cliff Writers labs is not in with the rest of the labs.

It's in a separate section he had.

And I think my first video lab experiments were in a different, they weren't in with the rest of the labs, they were separate.

Yeah, the labs are IJT files.

So you've probably just been overlooking them in your search mechanism.

Ah, that's the answer.

Yes, that would be correct.

And there are two forms.

There's the pre-805 form, I think.

And that's the old form that you could do a lot of different things, building labs, and there was a lab builder and a lot of things you did with it.

And then that was skinny down to a form that Eric developed that was just essentially a text file.

And it just, it's very simple, but it's very, it's actually super easy to develop for.

So that's why I think it's an area that hasn't been explored, because I think people got tied up in trying to make labs that were a bit too complicated, and they're hard to do.

Start with, but if you're trying to teach something, somebody about an application, you've got a lot of work to do that and build it up the right way, tell the right story.

But on top of that, there was a lot of equipment that you had to get working together to make it work.

And having a simple text file makes it a lot easier.

And the other thing I was going to say is if you did go back to primitives, and I understand what you're saying is that might be too low a level, but if you go back to primitives, you could have instances that were chapters.

So in other words, if I did a primitive, and I just wanted to know how take worked with, you know, rank one arrays, I could just do that.

And I could just say, hey, this is rank one, that could be that chapter.

But if you wanted to specifically know something about rank two arrays, or how does it work if it's boxed?

Or how does it work if this in different situations, you could have that as different chapters.

So you wouldn't be locked into going from the beginning to the end.

You could just go to that area.

And that might actually be a way to do it with J playground is that you've got a series of links that are identified.

So if you're dealing with something, and you say you wanted to work with cut, but you wanted to know something about how cut works, if it was, if it was given, you know, in case of minus three, where you're sort of tessellating, you just go to that section, and then it's going to give you a lab on that, that information, you don't have to wander through all the other things that could do.

It goes directly to the area that you might be interested in.

I threw into chat account, a list of repositories with counts for how many IJT files they have in each repo on in the J source.

GitHub account.

Thank you.

Yeah.

So I see there are 78 labs.

Donate official lab, but any lab labs.

Right.

Yeah.

Yeah.

So to me, that looks like J playground at least is doing IJT files.

Yep.

Whether or not those are IJT files that are the same are the string format or not, I would guess they are because that's the simpler one.

But I haven't looked into any more detail.

I thought that was.

Yeah.

And they are those files are line oriented, right?

I mean, they're.

Yeah, they're, they're, they're, they're, they're very similar to IJS files, except for they have a little more decoration, basically.

Okay.

So I can process them.

Yeah, you can, when you're doing an IJT file, you can, you can break it into sections and you can have text and then you can have areas that the user doesn't see.

Actually, I think you can still do that.

I think Eric was trying to take it out so that the user would see everything that was being done.

He didn't want to hide things.

But I think there's still ways to hide things.

Yeah, I think there was some of the stuff I've been doing with my video labs.

I had to be able to hide some of the things I was checking to see whether the system would work with what I was doing with the video.

Sure.

And then there was some little tricks I had to play with things too.

I was able to get.

That could have been done by having a, like having an add on that, that, you know, wrap those things in commands and then the person wouldn't, you know, they just see a check video or something like that, that would be, you know, not confusing.

Yeah, in effect, I kind of did a little bit of that.

I didn't make it an add on, but within my video files, I include the, the little functional things that I need to like run a video, like run a YouTube video and stuff.

I just included those with that file.

So they're, they're built into the file.

So each file becomes its kind of its own little add on that can run its video and it can run the video in J console as well, which is kind of cool because I figured out a way you can tell J console to launch webs, you know, a web instance, and then just run a YouTube window or YouTube video within the web.

And then you've got your console sitting on one side and you've got your image in your video playing on the other.

I guess, I guess the old the new labs just have string contents for the for the text, you know, zero colon zero, or the old one had nb dot and then a multi line comment section that wasn't really a comment and J followed by a close parentheses, some of the 00.

Well, and there's things you can there's different like I believe it's just going by memory, I forget is it if you start the line off with four minuses, it becomes a horizontal line.

It's stuff like that.

It had some had like a repair block and it had some assignments at the top is for that.

But the markdown kind of format.

Yeah.

I kind of like markdown, but it was, you know, J.

Sure, of course, of course.

And there's it's it's a mixed text and kind of like literate programming, maybe.

Yeah, we have mixed text and an executable code.

Although the, the more recent form, I think it might all be valid as a J script.

If you want to run it that way.

I believe it is.

I believe that was why, why, why Eric did it that way, is that you just you can just run it as a script.

But again, haven't tested it to see whether it works and I'm not aware of what playground is doing to run its labs, whether it's putting another layer on to be able to do step by step or not.

It's got a number of the old style labs in it.

So there's that.

And it's got a number of the new style to it for us.

I think what happens as long as the code is there for the old style labs, it will, it'll run because it's running the old style labs off the old code.

And the new style labs run off the off the new code and the biggest difference in the old style labs that I remember was it got very complicated in how you were going to display the labs you had available.

So the pop up windows and stuff like that became a lot more complicated.

And I think when Eric went to his new version, it became closer to what JHS does and what the iOS apps do is what they do is they provide you categories.

And then you actually enter the category in and then it provides the labs in that category.

And then you add you put in the number of that lab and then you're running the lab.

So a lot of that over structural overhead is how do we organize this whole file of labs doesn't exist anymore, but it's still there for the old one.

So you can, I'm not sure how much it would need to be used in playground because I think once you've got the corpus sitting there, you don't need to play around with that stuff quite as much.

But I know that was one thing it when I when I delved into labs spent probably a couple of weeks turning through labs to try and figure out how they work to see if there was any leverage I could do to use them.

And and what I found out was a lot of it was messages being sent from this which is organizing.

This is what a lab is and then this is what it does when it does this and this is what it does in this and then it checks back here to see if it is a lab and then it goes back.

It was it was a real maze and the new version is much cleaner than that.

It looks like the playground has a copy of the lab labs labs and also has another set of labs which are just its own collection.

Although some of those look like copies of the other set and some of them look distinct.

Well, if that was true, then it might be just as simple.

Well, again, those aren't being run from the editor, are they?

They're being run through the menu whether or not they're.

You know what I'm saying is what you would send into your your edit file that you would then load and then run the lab.

I'm not sure that there might be something have to be done before it would shows up like a lab on your display part.

Like to be able to go step by step.

Although it goes step by step line by line with the existing thing, doesn't it?

Or maybe not.

Because you can load all your you can preload all your information and then and then it exists in the other side to play around with.

Anyway, something I just thought of on my walk and it sort of fits to what you're saying.

And I.

I think something along that line would be good to go with the examples.

But I think the other thing that's that I find most useful about the GitHub application that you're working on is it gives you a way to measuring what's actually happening.

Like, you can say what we want people to do.

We would really like it if people program this way.

This would be much more consistent.

It would work really well.

And we can we can suggest this to people.

But then if you go back and you find nobody's doing it.

Then you kind of know it's for whatever reason it's not working, whether it's habit or whether it's if there's something you haven't thought through that other people go, well, I would never do it that way.

It's not useful.

It's also like a certain reassurance.

If you if you say, I think I'd like to do it this way.

If you can find that lots of other people are doing something that's just like that.

Yeah, you can proceed with confidence.

You're at least following the crowd, even if you're not necessarily doing the right thing.

But I think it also gives you a way to direct the crowd.

So you can suggest something that might be a new style and the example might be.

I know I talked to Henry in one of the episodes about fold and I said I'd spent some time playing around with fold and doing things.

He says, oh, good.

You, me and two other people are probably the only other people who even touch the thing.

And that was a couple of months ago.

And what that would be useful for is if you said you basically make some type of an educational campaign, you say people, this is really what we should be using fold for.

This is why it's so good compared to what you're used to.

It's so much more flexible.

It can do so many more things.

And then after that, if there was a way to analyze how people were using that code, you could go, yeah, it's being used.

This is effective.

There is a change happening.

The thing about fold is it's not one thing.

It's one, two, three, four, six, six, six things.

And you need to have not only each of those things has or some several of those things, you'd have a way of getting into it, of experimenting with it to build up your expressions.

That's what Jay's power is, is the ability to experiment and work with things.

And there are ways of doing that, but we don't have, maybe that's what lab needs to be.

A folds lab that shows how to build yourself up to the different kinds of folds, how to construct those things.

You say, yeah, this might be interesting, but I don't know how to get started.

Well, this is how you get started.

Well, and with fold, I can see ways that you could have certain examples.

Like for instance, things you fold can do that the other, our normal primitives of scan and the different… I mean, that part's the easy part.

The thing is, a person does an essay notice, say, fold with semicolon, right bracket for you and semicolon for V, just to see how it constructs or comma ampersand box, to give you an idea of what it's doing.

See what the process, what the data structures are like that it's constructing.

I was thinking more like you could have specific instances that everybody will at some point come across.

I want to do a scan, but I want to start with a specific value.

And I don't necessarily want to put it in my head on my list.

That would be awkward.

But that's something that fold does really well, really easily.

So that kind of an instance, you can start to… people who know, okay, I want to do this.

Why would I use fold?

I can do it this way.

Well, yeah, but you don't have to do it that way.

And this is a bit cleaner, don't you think?

And I think that that's kind of the motivation you might build some of those instances.

Yep.

And the other area that I know that it was really good in is, and it's essentially reverse scan.

You can create an effective reverse scan with fold.

So it works the way you would expect it to work.

Whereas in Dialog APL, and I think even in BQN, their reverse scan doesn't… it works the way, I guess you go, oh, yeah, I guess I see where that's working.

But it's not the way you think it should work.

Because it's not actually working back to front.

It's taking apart and then taking a bigger part and taking a bigger part when it runs through the… it's just fold actually, that's why it has the six instances.

It actually cleans up a lot of those issues and gives you explicit, this is how, why you're doing it that way.

But I just don't think there's a motivation for a lot of programmers to say, well, why would I need that?

I could fake it with this other thing.

And to some extent you can, but you actually can't do it well enough to really do what you think you're doing all the time.

How's the performance?

It's not as great as it could be, because right now it's implemented in J itself with several hundred lines of code.

So it's kind of more of a model at currently.

But still, there are contexts where doing a whole bunch of loopy code like that is better than a brute force technique that works around some other approach.

Yeah, that was one thing that Henry had said as well, was it's written in J, so it's not as performant as it could be.

But if enough people start using it, he said it's not that hard for it to be converted over into C code and actually run performantly again.

I don't think performance is what's holding it back right now.

Although in a way it might be if somebody was really wanting to use it in production, they would probably say, yeah, it's too slow.

I need it to be quicker.

But that would be enough.

I think if somebody were doing that and they talked to Henry, he'd say, well, that's enough for me to make it fast.

I can do that.

If it were more performant than the conventional techniques, that might also be a selling factor.

Yeah.

And I don't know enough about the implementation.

But because it breaks up into the different sections, it may be possible that it can be more performant than something that you would try and make up out of parts.

I was just responding to the point that performance isn't holding it back.

If performance were better, then you could say, oh, performance can really make it happen.

You could make that a selling point.

Yeah.

Yeah.

Yeah.

I think the area that I see that's underused with performance is special code.

But I think I mentioned that one of the things we were blue skying about, I was saying it would be great if there was a way to identify special code within a, you know, this sequence could be special.

Special code opportunities.

Yeah.

Yeah.

Right.

Oh, absolutely.

I know there's so much that I write.

If I want to make it fast, I have to go back and dig through special code to see what I'm doing and what might work.

And it makes a huge difference.

But it doesn't come naturally.

Yeah.

At least not for me at this point.

Another area that had come up with when I was working on the wiki is essentially what are breadcrumbs when you're going through the wiki.

So the old format would retain the file hierarchy.

So you would see you were in system slash installation slash wherever you were.

And if you wanted to back out, you could go back up and click on system and bang, you'd be back up at the top.

And the category tree that I've been doing, I've actually taken that away, which isn't a big thing for the category trees because the category trees can work back and forth in themselves.

And they'll always give you a link to where you want to go down.

And I think what I'm going to do is build links so you always can go back up the tree too.

But the question I've got is if you go to a link that isn't part of a category tree, it's just a link to something, and it's got the old breadcrumb, should I be putting in a category tree link as well because that's how you got here?

Because if you go back to the old system link, it'll take you back to the old home page or the old main page, which has the old breakdown of everything.

It'll seem completely foreign to where you were.

It sounds like those links shouldn't be there at all.

Well, and that's the other option is I take all those links out, but then I've got the responsibility for building the whole linking tree.

Well, that's true.

Yeah, and I'm not sure I want to take that on and not have it break and then you end up with a wiki that's not as effective as it could be.

So you're essentially mixing two wikis and the border between them is porous in places.

Yeah.

Okay.

And if you got to a page from a category page, the category tree page, then I would try and make sure there was always a link back to the category tree page.

So you have an option to get back there.

But there also might be a link to the old structure of the wiki.

But there's always a link back in the sense that there's a back arrow, right?

There's always a link back as a back arrow.

Yeah, you can always do a back arrow.

But it's not a breadcrumb.

No, but it seems like a lot less trouble for you than going and touching every page in this other older wiki.

I don't have to touch every page.

I just have to touch the page that are linked to the category tree.

Oh, I see.

I see.

I see.

You could go back and then go back to a guides page and then find something else.

No, I'm not sure I agree.

I mean, if you jump to a foreign page explicitly in a single jump, you're going to install a category back pointer there on that page.

Yeah.

But if the person links then to another foreign page, which you aren't linked to directly from your category wiki, he doesn't have that explicit category back arrow anymore.

No.

So he's just as stuck.

It just took him two steps to get stuck instead of just one.

Yep.

I think you set yourself an impossible problem.

Well, and the solution I was going to have to that is in the sidebar, I was going to have the main categories.

Oh, the sidebar of the whole forum?

Of the whole forum.

So you would have newcomers, developers, reference.

That would be on your side.

So you could always get back to that high level.

It's a little, you know, but it would be there.

Or there's also, we talked about the map.

There would be a link of the map of the hierarchy.

And that map would always be up in the upper right corner, just below the search, the current search.

And so you could always click on that map.

And the next step, you could go click exactly the part of the hierarchy you wanted to go to, and you'd be back there.

So you'd have that.

And that would be on every page, whether or not it was an old style or a new page.

So there would be ways to do it.

But I'm just thinking a lot of people tend to expect to find those breadcrumbs in the upper left, because that's where they're seeing the wiki develop them, as you go down the levels of the files.

And then the other thing is, when I was doing this, I was thinking, do I even want to take off those breadcrumbs at all?

Like I'm specifically taking, I have to put in a CSS thing on the category page to remove that.

Otherwise, it'll show up.

But if it shows up, then it shows up as a string that I don't really want to have there, I don't think.

It just kind of annoys to me when I've got the other links.

And there is a link that always shows up at the bottom of the page.

There's always a hidden category page that will show you the category that's hidden for you.

And if you need to get back up a level, you can always hit that and it takes you back up a level, because that's the category that'll be hidden to you.

And it's always there, but it's not as obvious, and it's at the bottom of the page.

So people don't know that.

Right.

I wish I had something profound to tell you about this, Bob.

I'm afraid I'm somewhat at sea.

I know, but I'm not expecting as much as, profound is good, but me voicing it starts to give me some ideas.

All right, well, good.

It tells me where there's uncertainty and where I might, you see, there are some areas that I might feel pretty sure that this would work.

And if there's uncertainty expressed, it's like, maybe you should keep thinking about that.

Yes.

So, sorry, go ahead.

I just think I haven't heard anything from Raul really yet.

I was still thinking about back on the folds and I threw a little comment in chat to correct a statement I made earlier and I just been thinking about folds while you've been talking about.

Sorry, I interrupted you.

I was going to comment on the comment in the chat, but it's been explained now so not necessary.

All right, well, I'll keep plugging away.

I would like to express, as I do every once in a while, my thanks to both of you for your continued support.

It really couldn't have happened without you.

I've lost track of most of my beta testers who had early enthusiasm, but for various reasons, I don't know why, have fallen away.

But you two have been.

.

.

We talk anyways, so it's easier.

I beg your pardon?

We talk anyways on these Thursdays.

True, that's a big part of it, I'm sure.

But for whatever the reason, I appreciate it very much.

So thank you both.

It raises a different question though.

You're right, the beta testers have fallen away.

And that's a concern to me.

Well, I'm going to do a round with them before general release.

So, sort of a re-contact note.

Hello, here are the nifty new features, you might want to try them out.

We're almost ready for general release.

Let me know if you run into anything.

Thanks very much, Ed.

Something along these lines.

He doesn't want anybody downloading the database right now anyways.

Exactly.

Right.

I'm in a very awkward and embarrassing position right now with respect to the database.

It's the sort of thing that would definitely get you not merely fired, but drawn and quartered at Amazon.

com.

There's no question about it.

They have a big release process where they just push back on the release if you're going to break it.

They've got a whole infrastructure.

They've got DevOps.

They're fine.

I'm just one guy and feeling my way.

I did, did I send a note out to you?

I did move to North America.

That doesn't seem significantly to improve download performance for me.

For whatever reason.

That's a good thing to check though.

Yeah, exactly.

I mean, I would have felt silly if doing it would have helped and I didn't do it.

And I guess that's all I've got.

I'm thinking just again, thinking along the lines of the beta testers dropping off, which it's a different situation because a user, you're not actually demanding anything of them.

You're just giving them a tool to use and then how they tend to use it is how they use it.

You don't know how much they're using it.

Beta testers are asking for a response.

And so you're asking more of them.

Yeah.

I'm thinking a big part of what we do with restructuring the wiki will also be introducing this tool.

Because honestly, this tool takes you a lot of places quicker in the wiki than a category structure can.

Category structure gives you a different tool to use.

You get a way of understanding it.

But to actually get around it, it's much quicker to use the search tool.

I, I, that was the point.

So I'm glad to hear that that worked out or seems to be working out so good.

I would be very careful about deemphasizing the web interface.

Yeah, it's what everybody can use.

I, the term lowest common denominator is an ugly one.

But it is.

And I think the last thing we should do, or you should do, is give up on the web.

If you see what I mean.

Yeah, no, that's a really good point.

But I guess what I'm thinking is it would be a mistake to introduce categories to the wiki, the changes we've done, and not mention that search tool.

Because to me, that would just be like, we're giving you half of the magic keys here.

And if you want to just use the web, that gives you an important leg up.

It should be better than what it was.

So good.

This is good.

But if we didn't tell you there was something else you could use with minimal need, but a bit more, but minimal need, suddenly it would open up a whole thing.

And I think the two work well together, but you're right, you still have to stay with the independent web experience.

Yeah, yeah.

So I think we're in violent agreement on this.

Yeah, but I think that will also help people continuing to use it.

And the other thing is, I almost, I didn't do this because I thought, no, this isn't fair to start firing, telling people they should get in touch with you about betas right now.

But the guy that I sent the, I think it was Elias, who's written the CAP language, and he was asking about the threads.

When I sent that link to him, I almost said, oh, by the way, I use this search tool to use it.

You might really, because no, but that's exactly the kind of person who would love that, because that's exactly what he wants, right?

He wants to dig through that information really fast.

And when he did it the usual way, he said, I couldn't find it in the wiki.

That didn't surprise me.

Even though it's threads, it doesn't show up as threads within the wiki.

It doesn't explain that way.

I mean, what is it?

I'm trying to think of the term that, is it pex or pix?

That's what Henry refers to them, pix.

Well, you could do a search for pix and it would probably take you there, but who's going to look for pyx?

Whereas threads takes you right to that whole conversation where threads were discussed and how do we implement this and what would we do with this?

And that's exactly what the guy was looking for.

That's great.

Yeah.

Right.

So we agree.

And I am, as I've mentioned, reluctant to release it because building is a lot more fun than product support.

But we're definitely getting close.

There's no question about it.

I've got lots of other things to focus on.

I love how that tool is developing.

I love the approach you're taking to the GitHub too.

I think there's a lot of potential there at a different level for a different group.

And it's funny, the same things you're doing with the wiki search and the GitHub is what I saw happening as well with the Playground.

And it's like my focus, what I've been told to do is look at the wiki and try and make the wiki better.

But in the process of doing that, there's these tools coming out that do make the wiki better.

But they're not the wiki, it's the structures around the wiki.

And it's every bit, and I think in a lot of ways, much more valid.

But you've still got the existing information in the wiki that's still that big ball of text.

But now you've got different ways of getting to it and interacting with it, which I think… In terms of the distinction between curation on the one hand, which is making the ball of text structure good and selecting appropriately and cross-referencing nicely and editing nicely and so on, that's all curation.

And you're doing that, and I have nothing to add to curation as far as I can tell.

But the reason I got into this in the first place, seven months ago, was that I thought that there was a role for presentation.

That there's a whole other independent axis of success for a corpus.

And that you could do a brilliant job on curation, but if you presented it in a 200-meter scroll of vellum, nobody would ever try to use it.

So beautiful curation, lousy presentation, and it's a failure.

And I really do the web as suboptimal for presentation.

It's slow and awkward, and you generally have an ant's eye view of things, and there's a lot not to like.

And my goal was to get away from the 200-foot scroll of vellum, to get away from the web, and try to come up with a presentation mechanism that would help the curation shine.

Yep.

And I think you've done it.

It's just a matter of knocking the edges off so you can actually get it out.

Yeah, so it works.

Yep.

No, it's good, and it's going to make a big difference to how people perceive the wiki as.

.

.

And I think at this early stage, reflecting on what we were talking about with developers and stuff, I can see that making a big difference with the GitHub search too.

I can see that being a very powerful tool for somebody who's looking to develop in J.

I'm looking forward to getting feedback on that.

First from the British APL Society.

Well, probably, they don't meet for another month, so it'll probably go out well before then.

Oh, I think they're every two weeks, aren't they?

Are they every two weeks?

I think so.

Yeah.

Interesting.

Okay.

They were alternating with dialogues webinars, so next week will be.

.

.

Morten Kronberg is talking at the next one.

I forget what the topic is.

And if they do decide that they're going to do.

.

.

Like I would say probably for the past four months, all they've done other than their AGM was open sessions.

And that's probably why Adam was doing the Q&A with them.

Because quite often if he's there, they'll ask him questions about how would this work and how would that work, and it becomes a big Q&A session.

But if somebody else has been tagged to present, then that's probably not the best time to think of getting extra time on it.

You might get extra time.

No, I wouldn't try to compete with Morten by any means.

But he's a week from now, I believe.

Okay, so that's what they're going to do is that'll be the dialogue webinar.

It'll be Morten.

Yeah, and if they're into another open one, the only thing I could think is it's usually Jake or Ziggy are the two that tend to run, like host the whole thing.

And it's.

.

.

There is a link, there is an email if you want to get on the agenda.

So you can do that in advance.

And they're really good about that.

And if you've got something, I mean, honestly, that kind of a GitHub thing, I think they'd be really intrigued by.

Even more than it affects J, but also how it might affect APL.

Well, and it's all inspired by Applecart via Raul, who first turned me on to the idea of that style of interface.

We don't have 6000 curated J sentences with comments the way Adam has put together 6000 curated APL phrases with comments.

But we can use a similar style of interface to get access to the corpus.

And I can see Adam being very interested in the approach you've taken, because in a lot of ways, it would open up opportunities for him and not needing to, you know, be as much hands on because you just look at the code body.

And that's what's going to happen.

And you don't need to curate that.

It's just what is right.

The big question is large.

Right now you're using J sources, your code, right?

Your code body.

The J software GitHub account, which is about 120 repos.

Yeah, okay.

It's about 3000 files.

But then the next question is, how easy is it to incorporate to bring in another repo or make make it custom?

So you if you were doing your own repo, you could you could drop it in.

Oh, that's just a matter of writing a crawler that will adapting a crawler that will go against whatever repo you're interested in.

Yeah, because you see, that's the area that I think that could actually be a probably a commercially viable product for anybody who'd be developing.

You drop in your repo and then just have it.

Yeah, maybe.

I don't know what this I'm not.

I guess you do do code search with Google.

If you're searching for Python for Python code, for example, you can find samples that way.

You usually wind up at stack overflow.

And pick up your examples there.

The challenge with J, of course, is that punctuation tends to be discarded by indexers.

The challenge with APL is obvious.

I don't, I don't even know if there is other than Apple cart.

Is there a search mechanism for APL?

Do people search APL repos?

They must.

No, I'm sure they've got them.

So they must have a way of looking at them, right?

You would think.

And to me that especially if you in a large organization and you were trying to onboard somebody That's probably the quickest way to say this is how we deal with this right?

That's what I'm thinking is a way to I mean, if you just confronted the J software account.

And your, your goal was to learn about J from that just by using the GitHub, you know, web interface or any GitHub client any get client to learn about the code, you There's no way.

Forget it.

You know, you bring up a file and read it and then another file and read bits of it.

And it's just, it just strikes me there might be there must be better ways of reading code in a large repository more instructive more Higher information content ways higher bandwidth ways of reading code.

Once you've got a lot of it.

If there are patterns in the code that you can spot and highlight and explain And I think, I think there might be something we could do with comments like right now the There's no sorting of results in the GitHub search, but a smart way to sort results might be right of the lines that have hits, which ones have comments.

And which are the longest comments.

All right, we'll put those hits at the top because it really isn't right now.

It's just sort of based on where they occur in the repository and sequential order.

But if there's a if there are comments that's possibly not probably possibly more helpful than a line without a comment.

So that kind of thing.

I don't, I'm not, I'm good.

I don't know, but I just think it's potentially a fruitful area for exploration.

The other thing that I thought of.

I just thought of, and I think it applies more to coding styles is if you took say The code for dissect as a source.

It's coded quite differently than what the standard library of J is because one one's coded by Henry rich and there's huge amounts of comments and it's actually quite an amazing Scripts he's done there.

And the other was, I think, Chris Burke wrote and he was just writing it to fill in the gaps because there was nobody had actually taken All these little verbs and yeah you know applications and conjunctions that people were using and actually codified them into a library.

And that's what he did with the standard library.

So two different individuals, but I don't think either of them tend to use tacit And so again, you've got if you if you chose different libraries, you could actually then steer people into different styles of coding to That's true.

Yeah, that's true.

And if you could, if you could tell at indexing time.

Which sentences were tacit and which were explicit, you could put a filter on and say I'm only interested in tacit results.

Yeah, that's that actually is kind of hard to do, though, because of the mix of tacit and explicit right within it was an explicit code, you'd have to have lines.

Yeah, but it's okay.

You don't need to exclude you don't need to exclude whole files or include whole files you're excluding or including individual lines from the results.

So, a lot if you're looking at a definition of a verb, which is what most definitions are anything that ends with X or Y on the right hand side is is unless it has big parentheses in there is is basically explicit code.

But big parentheses also are the thing that you know we have big parentheses without labels inside that it's going to be tacit basically You don't need to be right.

You can have a mix is what I'm saying some lines are both That's okay.

That's okay.

And you don't need to be 100% right.

You just need to be better than written significantly better than random Or you find a GitHub repository repository of somebody who writes tacit code.

And you link that one in and that's that becomes if you're interested in tacit centrally you could have.

Do you want tacit or explicit answer.

Yeah, right.

That will be the filter.

Yeah.

But anyways, as long as a sentence or a parenthesized expression with a noun on the right hand side is going to be not usually not tacit sometimes you can have it passive because it's a conjunction or something, but usually it's not going to be tacit Okay, I'll think about it.

I have actually other much more pressing problems.

I created for myself.

I have to address at this point.

Okay.

Yeah.

Anyway, I think that probably wraps this up.

And as usual, thanks a lot for the discussion because it's very helpful and and I find quite stimulating so Good.

I enjoy it myself.

And I Again, And one thing, Ed, I think you've got a unique combination of attributes because you have a level of persistence that most people who have are able to delay gratification and you seem unable to delay gratification, but have high persistence.

Like you want stuff to happen now.

Oh, well, that's true.

But you persist I try.

Retirement helps.

I commend it to anyone who's interested.

Yeah, yeah.

It, it opens up your life a bit if you choose to have it that way.

Yeah, it does.

Yeah.

All right, gentlemen, take care.

Thanks again.

Okay, take care this week.