I'll start, when I click in this http://127.0.0.1:8000/a/thread/8144401128cb2f98544332c712ef7ea5d998b501e770599950d6927ee950cd92.html thread on link in #b0e93417 message that has reply to other message, instead of just jumping to it - I jump and page reloads. I can add that using just #anchor link to element with id (instead of script) - also allows to send links to post in thread natively by appending anchor link to thread link. Which is not usable now. Also need catalog...

>>5208e11d I'll work on this.

a /unread/ board that gives you all of your unread threads, or some kind of way of seeing how many unread threads i have, and where.

Having useful information in title like board + thread name would be nice

I mean the tab <title>

image.webp, 3.82 kB

there seems to be a .gitignore missing for installed modules and other misc. files

>>b5987a7d Added one. Are you thinking of making a merge request for something? Especially with UI and QOL features i'd be glad for any.

>>727724dc >>78dd9186 Added this.

vergil_looking_at_laptop_confused_[meme].png, 338.92 kB

>>e8234e33 i'm probably too retarded to help in any meaningful way

links.png, 274.11 kB

>>0ea93f83 never say never! also latest version supports hyperlinks

When I click on post ID to reply while being at board index page, it adds link with that post ID in message field - but does not put "replying to ..." into subject

having the tsconfig.json options toggle-able in the home menu would be nice

>>dba8c44f good point, probably either it should jump into the thread or allow you to reply to a thread while still in the index >>5169f4e2 the config.json right, yeah that would probably be good. something i was thinking of is maybe it'd be cool if there was a way to restart the frontend/pageserver while keeping the peerbit node online.

>>5208e11d This should work properly in the latest version.

Have you ever heard about dollscript and similar userscripts? My main interest is subscription to threads and automatic update of unread messages counts for each thread in docked menu. Maybe it can support our frontend with some modifications

>>337ee301 I haven't heard of those, are they something you can use with tampermonkey? That would be convenient, and is definitely possible, there just needs to be some sort of event emission when a new post is successfully validated, then that can be connected to the frontend somewhere. Then actually we could also do live auto updates of threads like jschan has. Where do you think would be a good place to put the indicators?

>>5a141802

>are they something you can use with tampermonkey?

Yes. By the way, they even have issue to support jschan engine https://github.com/SthephanShinkufag/Dollchan-Extension-Tools/issues/1395 but with no activity for last 3 years

>Then actually we could also do live auto updates of threads like jschan has.

That would be great

>Where do you think would be a good place to put the indicators?

Maybe just mimic dollchan design, so docked at the bottom edge of viewport

>>bb793b82

>Maybe just mimic dollchan design, so docked at the bottom edge of viewport

or maybe at right sidebar since there's unused space

1650506083776.jpg, 1.44 MB

>>dba8c44f >>a41ef14e This should work in the latest version; if you click on a post hash from the index page it will jump into the thread and autofill the >>hash in the reply box.

Proposition: optionally create pair of keys which can be used to natively sign own messages and embed public key + signature in some metadata field like "Name", thus authenticating your message (proving your identity). Basically crypto-tripcodes via pgp or something

How do I update without purging config/watchedBoards.json?

Also, pulled new commits, everything works but I have

>{"level":40,"time":1711472992618,"pid":30593,"hostname":"disp448","module":"client","msg":"Failed to dial bootstrap address(s): \"/dns4/3f22709abb2362bf007d21164f6b2ef0237b03b8.peerchecker.com/tcp/4003/wss/p2p/12D3KooWGH8Fg1z1wW6KXgT2ngNr6cR69t47cPXvmSmmVj3nKyxb\". Reason: AggregateCodeError: All multiaddr dials failed"}

some instances of

>Error: File size exceeded max size of 4194304

and

>{"level":50,"time":1711473161092,"pid":30593,"hostname":"disp448","module":"shared-log","msg":"Failed to find peer who updated their role: Stream to P8xNaszfuq4rEZQc5mb2u5ccFQsnCC46+KPd9mG8RcM= does not exist. Connection exist: false. Route exist: false"}

and lots of

>{"level":50,"time":1711473162161,"pid":30593,"hostname":"disp448","msg":"Failed to resolve all heads"}

>

Also I suggest .post, .post_reply {word-wrap: break-word;}

>>d636261c Actually this is already built-in to peerbit; each post is signed with a keypair. For the sake of anonymity, we use a random throwaway keypair every post, but this can be disabled in config.json, so this is very do-able. Someone also suggested being able to encrypt posts to other people based on their keypair, which seems cool. >>69e05fac Can add this to the .gitignore file. >>2b98688d One of the bootstrap nodes has issues occasionally.

>Failed to find peer who updated their role

This is because a peer went offline while data was still being transferred, the error handling for those types of situations is not so polished yet.

>Failed to resolve all heads

This is a result of other people deleting things when you don't consider them moderators, they broadcast their deletion actopm but it doesn't get validated by your client. >>c92001ff good idea

>>69e05fac action*

function isElementInViewport (el) { var rect = el.getBoundingClientRect(); return ( rect.top >= 0 && rect.left >= 0 && rect.bottom <= (window.innerHeight || document.documentElement.clientHeight) && /* or $(window).height() */ rect.right <= (window.innerWidth || document.documentElement.clientWidth) /* or $(window).width() */ ); } document.querySelectorAll('.post-quote').forEach(quote => { quote.addEventListener("mouseenter", (e)=>{ quote_target = document.querySelector('div[id="'+e.target.getAttribute('hash')+'"]'); if (quote_target == null) return; if(isElementInViewport(quote_target)){ quote_target.style.filter = 'brightness(2)'; } else { quote_target_copy = quote_target.cloneNode(true); quote_target_copy.id = quote_target_copy.id+'_copy'; quote_target_copy.style.position = "fixed"; quote_target_copy.style.top = (e.target.getBoundingClientRect().top+10)+'px'; quote_target_copy.style.left = (e.target.getBoundingClientRect().left+10)+'px'; document.body.appendChild(quote_target_copy); } }) quote.addEventListener("mouseout", (e)=>{ if (quote_target == null) return; quote_target = document.querySelector('div[id="'+e.target.getAttribute('hash')+'"]'); if(isElementInViewport(quote_target)){ quote_target.style.filter = 'none'; } else { document.querySelector('div[id="'+e.target.getAttribute('hash')+'_copy"]').remove() } }) }) If anyone can do better - go on Also post favicons

>Someone also suggested being able to encrypt posts to other people based on their keypair, which seems cool.

Are there any gladden threads I'm not aware of?

>>13d7de15 Oh this was on lambda/tech/

>>6b40fbe5 z5lcip4dafatwwa6hvyibizpzwycvwp67cjga3hzjhxhwvuyaqavxnid? Can't load it since I saw gladden thread tho

>>84aa1f41 I think it was down for a little bit there.

>>c92001ff Added this.

>>8a6be4b0 this is really nice, thanks

image (1).webp, 5.18 kB

>>3a1ecf70 maybe format this to a human readable format

>>f5c7eb5b forgot the ?

Maybe if file size (or amount of files) fails validation, this shouldn't be the reason to hide entire message? Only replace file with red cross or like this (people who don't see what files they're missing have no reason to mess with their config + why hide message contents if text passes validation)

>>2330eb35 (1) On the receiving end, there's no need for the files themselves to be valid in order for the post to be valided per se (except for the reported file size (in the file reference part of the post, not the file itself) not exceeding the limit. That can be changed interdependently of the size of the actual files. (2) Separately (but using the same validation settings), when posting via the UI your client checks the size of the file you are about to post, and if it doesn't pass your own validation criteria size limit, it stops the posting. That itself could be turned off but then people could post posts with files that their own clients wouldn't accept. I guess the main thing for what you're saying is maybe actually the former (1).. just disable the file size reference limit on the post itself, so the post can go through, and people would see the size in the post and see that it's above their configured limit. I think editing post.files.size.maxSize in config/validation.json to a very high value would make this work (at least on the receiving end- you could see posts referencing too-big-for-us files)

>>2330eb35 >>517f9168 Disabled this check in the default validation configuration. Even disabled it shouldn't be a big source of mischief, as you can still only store a single bigint in the size field in question.

>>49c9e102 For the filesize anyway. For the number of files have to think about it a bit because it's an array so unlimiting it outright could open an attack surface for people to stuff lots and lots of data into posts. Maybe a soft limit and a "hard" limit type of thing of like 100 files.

>>2db2a01c >>49c9e102 >>517f9168 Thanks, I think that's a good change.

c413754755a6bf7366e255eadc7b6689-405102618.png, 92.82 kB

>>eaee34b3

just added the ability to configure the startup homepage if you want it to open eg. the overboard by default on startup, and now you can do /b and it will take you to /b/index.html instead of erroring.

Can we get images previews to be a link? Ctrl+click = image in new tab Middle Mouse Button click = image in new tab Click = enlarge your picture

also BB-codes or markdown or something

>>a93d1d59 bloat

>>8722cbbf I'll take a look at this

not sure what's up but my version of /test/ is empty. deleting doesn't work in the overboard, it would be nice if it did, also I find the mechanism uncomfortable a good ol button would be nicer. can I undo deletes?

mb /test/ just took longer than I expected to load it's fine now.

>>87b8251e I'll look at deleting from the overboard.

>can I undo deletes?

Not currently, as deleting removes the data from your disk and marks the hash of what you deleted with a record that the item was deleted and that you don't want to re-receive it from anyone. We can add something to erase that deletion record though and so your client should get it again from other peers. But there won't be a direct record of what the actual item was that was deleted (aside from the hash) so how to keep track.. hmm..

>>d70636ec makes sense the only reason I wanted to undo is because I was testing lol, is there a file I can remove to reset deletes? I don't think it's necessary to add a way to undo just dont delete stuff you dont want to do delete.

>>9395972a

>is there a file I can remove to reset deletes?

Not something that's clearly separated from other data. The only way would be to delete /storage and reboot and then everything will be fetched from other peers again.

>>9395972a Wat quote didn't work

>>82816ea8 Oh nevermind just a css style thing

>>87b8251e fixed deleting from the overboard

It would be handy to have threads=N URL parameter for overboard as default is too low. Also, about merciful way of showing too big files and posts with too many files. So I posted a h/thread/268ed9eddc1ec51d847487dedc756ccca9dd6838c9e96621905cd050acd4ec51.html (I don't have SFW example for this bug, sorry) and appended a few posts with 4 images each (which was ok with my config). Now I'm on a new version and there are no those posts, but first message in thread is there and someone's reply is there too. I'm not complaining, just curious. Maybe I was last peer who has those posts, and after I deleted VM with gladden, there were no peers to load posts from. But the thread is there! How it may be that someone got thread but not posts, they are loaded together even if you don't enter the thread. Basically I'm wandering if propagation is ok for smallboards\private boards. Maybe it would be reasonable, if you create privateboard with low peer count, to also have super-seed sort of server as a backup for this board, to avoid situations of loss info because of low propagation. Also if you use .sidebar { min-width: var(--sidebar-width); width: auto; } then sidebar will adjust to message field when you resize it manually.

>>86aa6220

>How it may be that someone got thread but not posts, they are loaded together even if you don't enter the thread.

Okay maybe they were just invalid for their config at the time of load so not propagated and not loaded, but first message and text-only post were valid for that person. So that issue should not replicate again, nevermind

OIP.jpeg, 12.92 kB

>>86aa6220

>mfw peer to peer hentai

>

>>86aa6220

>It would be handy to have threads=N URL parameter for overboard as default is too low.

Noted, the way things are currently set up this is simple to implement. For now you can also modify threadsPerPage in config.json and it should affect this. For boards with less users yeah you need someone to be online at a given time o be able to propagate the posts/files to them. I know there at least two seedbox nodes like you described. I think /h/ is covered by the set of the seedboxes, but it's possible they may not be updated to the latest version that supports multi-file posts yet. And yeah you need to make sure that board has seeding for when you're offline for low peer, though if you're not both online but then happen to come back online at the same time later, the content should update for the other peer appropriately. There is some variable time to replication it seems like, so maybe yeah you deleted the VM too fast for it to transfer.. or maybe it was a config issue like you said (but now we support up to 64 files by default so.. I wonder). If you have a private board or particular board you want me to add to my own seedbox node (the one on the gladden.peerchan.net server) let me know and I can seed it.

>.sidebar

Nice ty. Wrangling the width of the sidebar and other elements in relation to it has been a challenge lol. >>360ee0d9 Now yer getting it.

posted 53 files, 3 shown, maxFiles is 64, what's the problem? http://127.0.0.1:8000/mu/thread/dc043c7eeecc08bd7df97fd94d3a8d5932d99aee26e59eb63b20d89f8b317902.html

"maxFilesPerPostToShow": 3 uh ok I forgot

I posted a few files (maybe like 200 = ~800MB) on /mu and now node process eats 4Gb ram. Had to unsub, ram consumption is back to 675Mb. Now what? Newest version used.

>>38a21637 So you unwatched and rewatched /mu/ and the RAM is back to 675 mb? What was the exact process you did?

>>2c58b4fd I unwatched. I'll rewatch it now

>>2c58b4fd Also I did an update recently to avoid storing the file chunk contents in RAM: https://gitgud.io/threshold862543/gladden/-/commit/a84042b30d3445b34569026251919d5b2110e035 so even if you have a lot of files being hosted at once it shouldn't impact the ram too much as they'll only be on disk and will be retrieved as needed (that's the idea anyway). There are probably other factors that contribute to the RAM usage than just the index sizes, but in a pending update to peerbit the post/file indexes themselves will also be able to be stored on disk, should also reduce RAM usage: https://github.com/dao-xyz/peerbit/pull/287 https://github.com/dao-xyz/peerbit/issues/274 >>74767a7d Oh okay, hopefully it doesn't crash.

>>0d15b982 It might also be that the write itself was okay, but having lots of audio files on the same page all attempting to loading a buffer type thing from the backend at the same time, leads to the issue.

>>0d15b982

>Also I did an update recently to avoid storing the file chunk contents in RAM

I'm up to date

>Oh okay, hopefully it doesn't crash.

I killed it at 3.5G ram consumption in process of /mu adding, so no

kinda funny to have special board to crush gladden but it seems that any board can (and will?) be used for that with enough files posted

>>e6a7659d From my point of view your posts are visible on /mu/ but the files are red Xs having failed to load. Hmm so maybe there should be a way to prune local content. And hopefully that peerbit upgrade will affect this positively.

>>8f7be9af I mean, to prune local content on a given board, rather than having to delete storage and wait to rebuild your local databases from the other seeders.

I could live without /mu for months till update, but I still don't get exact reasons for that ram usage, and if other boards will take more ram the more files they have\the more files you post. This could also be used to effectively turn off gateways though

>>25f2fedb It seems like it uses more RAM than the file size itself. There are two things that happen with files, one is that the searchable index is held in memory. In theory the ram usage of this should be mitigated by https://gitgud.io/threshold862543/gladden/-/commit/a84042b30d3445b34569026251919d5b2110e035 as only the chunk metadata (not the up to 1mb of actual chunk data) is kept in the searchable index. But there is probably additional overhead with this on a per-document basis. The second thing, which I'm guessing is the source of the issue, is the replication layer. In order to transmit all that file data to peers, your node loads it into memory at once in preparation to send to others. Here the indexer is irrelevant because the full chunk data must also be shared. And there is also probably overhead associated with this. It could be that if you send it to multiple peers at the same time, there is additional ram usage, but I'm not sure about that. It's definitely something that should be fixed. I wonder if configuring fewer files per post would help... it doesn't seem like it would directly though, but maybe breaking files gulps into smaller chunks could help avoid overwhelming the replication mechanism. With gateways, the nginx config of gladden.peerchan.net does have an additional cap on the size of the request body which could help mitigate this but yea lol.

>months

One thing you can do in the meantime is delete the storage folder and reboot, you'll get a new client ID and everything will be regenerated from the swarm. Or I can also set up a "prune board data" function and that can be used for just /mu/. Gonna keep thinking about this.

>>25f2fedb Also ideally there would be the granular replication control but maybe try setting replicationFactor in config.json to null and seeing what happens when you try to open /mu/

>>e41b0663

>try setting replicationFactor in config.json to null

Will that make me a leech lol? Anyway, tried (without any other modifications like clearing folder and rebooting) and repeatedly failed to even start: Starting Server at 8000:127.0.0.1 Successfully initialized Peerbit node. %my peer ID% {"level":40,"time":1713516027776,"pid":1054728,"hostname":"host","module":"client","msg":"Failed to dial bootstrap address(s): \"/dns4/3835c3783230009a90ae4f8acc7ee22bee09e317.peerchecker.com/tcp/4003/wss/p2p/12D3KooWL8xa855uAcLpstvXSyzVqUgErW5kkwJ2TSexm4EnT3Qg\". Reason: CodeError: The operation was aborted"} Failed to bootstrap: Error: Failed to succefully dial any bootstrap node at Peerbit.bootstrap (file:///home/user/gladden/node_modules/peerbit/lib/esm/peer.js:203:19) at async Module.bootstrap (file:///home/user/gladden/dist/db.js:122:5) at async file:///home/user/gladden/dist/server.js:1264:13 Opening database for /landing/... { replicationFactor: null } Opening database for /help/... { replicationFactor: null } Opening database for /a/... { replicationFactor: null } Opening database for /b/... { replicationFactor: null } Opening database for /4chan/... { replicationFactor: null } Opening database for /тест/... { replicationFactor: null } Opening database for /pics/... { replicationFactor: null } Opening database for /fap/... { replicationFactor: null } Opening database for /tech/... { replicationFactor: null } Opening database for /bb/... { replicationFactor: null } Opening database for /ru/... { replicationFactor: null } Opening database for /g/... { replicationFactor: null } Opening database for /blackhat/... { replicationFactor: null } Opening database for /psy/... { replicationFactor: null } Opening database for /v/... { replicationFactor: null } Opening database for /h/... { replicationFactor: null } Opening database for /dr/... { replicationFactor: null } Opening database for /undefined/... { replicationFactor: null } Opening database for /pol/... { replicationFactor: null } Opening database for /gladden/... { replicationFactor: null } Opening database for /wp/... { replicationFactor: null } Opening database for /Λ/... { replicationFactor: null } Opening database for /λ/... { replicationFactor: null } Opening database for /soc/... { replicationFactor: null } Opening database for /youtube/... { replicationFactor: null } Opening database for /biz/... { replicationFactor: null } Opening database for /lain/... { replicationFactor: null } {"level":40,"time":1713516027777,"pid":1054728,"hostname":"host","module":"client","msg":"Failed to dial bootstrap address(s): \"/dns4/3f22709abb2362bf007d21164f6b2ef0237b03b8.peerchecker.com/tcp/4003/wss/p2p/12D3KooWGH8Fg1z1wW6KXgT2ngNr6cR69t47cPXvmSmmVj3nKyxb\". Reason: CodeError: Could not connect to /dns4/3f22709abb2362bf007d21164f6b2ef0237b03b8.peerchecker.com/tcp/4003/wss/p2p/12D3KooWGH8Fg1z1wW6KXgT2ngNr6cR69t47cPXvmSmmVj3nKyxb"} Failed to open databases. Error: Unexpected, sortedPeersCache is undefined at SharedLog._modifyReplicators (file:///home/user/gladden/node_modules/@peerbit/shared-log/lib/esm/index.js:927:23) at SharedLog._updateRole (file:///home/user/gladden/node_modules/@peerbit/shared-log/lib/esm/index.js:208:40) at SharedLog.afterOpen (file:///home/user/gladden/node_modules/@peerbit/shared-log/lib/esm/index.js:431:20) at async Documents.afterOpen (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/program.js:129:13) at async FileDatabase.afterOpen (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/program.js:129:13) at async PostDatabase.afterOpen (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/program.js:129:13) at async fn (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/handler.js:131:13) at async file:///home/user/gladden/node_modules/p-queue/dist/index.js:187:36 An uncaught exception occurred: Not initialized Stack trace: Error: Not initialized at get topic [as topic] (file:///home/user/gladden/node_modules/@peerbit/rpc/lib/esm/controller.js:284:19) at RPC.getTopics (file:///home/user/gladden/node_modules/@peerbit/rpc/lib/esm/controller.js:289:22) at file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/program.js:137:38 at Array.map (<anonymous>) at SharedLog._emitJoinNetworkEvents (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/program.js:137:14) at node.services.pubsub.addEventListener._subscriptionEventListener._subscriptionEventListener (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/program.js:118:76) at [nodejs.internal.kHybridDispatch] (node:internal/event_target:786:20) at DirectSub.dispatchEvent (node:internal/event_target:721:26) at DirectSub.dispatchEvent (file:///home/user/gladden/node_modules/@libp2p/interface/dist/src/event-target.js:43:30) at DirectSub.onDataMessage (file:///home/user/gladden/node_modules/@peerbit/pubsub/lib/esm/index.js:446:30) zsh: exit 1 npm start (subtle board list flex) Now back on 1, I note that /landing and /b take 1h+ (sic(k)) to sync (being in closed status), even while not having much content. I was able to post on other boards. And they fail to open before sync finished, with Failed to get posts for board "landing". TypeError: Cannot read properties of undefined (reading 'documents') at Module.getThreadsWithReplies (file:///home/user/gladden/dist/db.js:260:50) at file:///home/user/gladden/dist/server.js:864:66 at Layer.handle [as handle_request] (/home/user/gladden/node_modules/express/lib/router/layer.js:95:5) at next (/home/user/gladden/node_modules/express/lib/router/route.js:149:13) at Route.dispatch (/home/user/gladden/node_modules/express/lib/router/route.js:119:3) at Layer.handle [as handle_request] (/home/user/gladden/node_modules/express/lib/router/layer.js:95:5) at /home/user/gladden/node_modules/express/lib/router/index.js:284:15 at param (/home/user/gladden/node_modules/express/lib/router/index.js:365:14) at param (/home/user/gladden/node_modules/express/lib/router/index.js:376:14) at param (/home/user/gladden/node_modules/express/lib/router/index.js:376:14) buildIndex: 1.853ms Tried to reload /b but got Error reloading board: Error: Program at zb2rhcYxsgmKoRRwpsxRGXr7uwuC7dz3k74uc5zE3SHZxJqQJ is already open at ProgramHandler.checkProcessExisting (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/handler.js:38:23) at fn (file:///home/user/gladden/node_modules/@peerbit/program/lib/esm/handler.js:106:41) at async file:///home/user/gladden/node_modules/p-queue/dist/index.js:187:36 After 1h wait I restarted. And now /landing, /a, /b, /pics, /fap, /ru, /g are closed (for 10min+ and probably indefinitely) Wtf? Alright I just load gladden from other VM. I posted on /landing fine, before I fist changed that param to null. I keep running faulty VM in case additional info from it will be needed for debugging. I'm on last version currently. Also, bug with cyrillic (and i guess other non-latin) filenames: http://127.0.0.1:8000/%D1%82%D0%B5%D1%81%D1%82/thread/7211a393c3c5ad404cd8d44ebca2f94eb8da8a3033fc0181948e11104716dc0f.html#c5db7d04bff35f719878e89c676ad41418f933c637f4617f2d66c3c460476ecf

>>2df536a0 Thanks for testing and the error logs.

>Will that make me a leech lol?

Nah in this case not setting it will allow the default sharding algorithm to kick in where you seed a variable fraction of the data based on the number of peers connected. It's possible to configure yourself to act as a leech tho, but I didn't add a config option to do that easily for now. Okay so latest version even after this https://gitgud.io/threshold862543/gladden/-/commit/2b007c9111e4b4c9222c875240ef18db4902e95b hmm I guess I didn't fix it correctly. Maybe I should try to break things in the same way and see if I can replicate it so I can also test the reload function in an actual crashed condition as well I'll check up on the filenames thing too.

Patched the filenames thing so it should allow unicode.

Seems like there's quite an overhaul of Peerbit's database backend coming actually, optimistically this will solve the /mu/ issue and also minimize problems in a future case where there's a lot of data being passed around at a rapid rate.