The most recent 20 or so dev-diaries are on my website now. I found myself wanting to refer back to them sometimes, however couldn't figure out how to use the tag plugin in ssb, so they're hosted with the copy and paste method for now.

Also has a selection of photos from eventual gram, which is the ssb client thing I've been making. I need to finish up with working on it actually. Not for any real reason, I just feel like I'm done. A cool thing about it is that it works with the existing pubs. I used the same caps & sign values that are in patchwork, not sure if that is necessary regarding the pubs.

My attention has been more focused on ssc lately. It's a collection of functions for working with merkle DAGs. It's based heavily on ssb, but with a focus on separating policy from mechanism as they say. Meaning that nothing in here should do IO, it's all in-memory functions. There are several libraries for merkle-based replication (this and hyper*), but nothing that is a nice building block, without a disk IO element, that I can see.

And also ssc-pub, which is like current pubs, but built with different stuff. Not sure how I feel about it. There is just more vendor lock in this way. But it shouldn't be too much work to set it up. The existing ssb pubs work without redundancy or uptime concern because it is a gossip network. Nonetheless, it is cheaper to use lambda functions and a DB service I think. Have also looked at cloudflare, which has similar stuff — a KV store and something called "durable objects", which I'm not exactly sure what it is, but they enable websockets which is cool and new.

Also made a page of examples of things. It's just cool stuff I saw on the web. I think mostly it is from https://www.cassie.codes/ , a classic good website.

It seems like I'm getting much more gossip now on ssb. Maybe people are just more chatty.

#dev-diary Have been thinking about git/merkle-dags/ssb today. I was wondering does git sign commits? I know that is the premise behind ssb, everything is signed and also a merkle-tree. I found this fantastic article about it: https://withblue.ink/2020/05/17/how-and-why-to-sign-git-commits.html .

So no, by default git does not sign anything, it just creates a merkle-dag, where every commit is a hash containing the previous commit (i think).

I think this train of thought started by thinking about more general ways of doing ssb. The technology is pretty generic after all -- hash functions used in a particular way. It's kind of a shame that it feels so tied to a particular implementation. I guess the database/user-id and replication are kind of coupled at the moment.

Anyway in the article they mention something called gpg, which I think is a general unix way of doing digital signatures (like we have in ssb).

It makes me wonder about having a ssb client where you can choose an identity, like using your gpg credentials, or metamask or something. Digital signatures have gotten more attention because of bitcoins maybe.

Just found out about the service fission today. I have read about creating accounts so far. It reminds me a lot of ssb in the sense that an identity is a pub/priv key pair. However, there seems to be more built around the id keys, for example you can link multiple devices or use it in a browser.

What is fission? I still don't know.

Fission Drive lets you browse public, unencrypted files stored on IPFS.

So it uses IPFS. Is it a service that 'pins' your files also?

You can browse any IPFS directory, not just those hosted on Fission, by entering in the hash, or Content ID (CID), of a directory.

So I guess it is a host/'pin'

Your browser holds the unique, private key to this account, so you don’t need a password. In order to not get locked out of your account, we recommend that you link to at least one other device, like your phone, tablet, or other computer with a web browser.

That's kind of cool, using a second device in case of account lock-out.

I see there is a dialog for permission to access the file system from the browser. How does that work? my understanding was that websites are never able to do fs access.

Found a repo https://github.com/fission-suite/webnative

The Web Native File System (WNFS) is built on top of the InterPlanetary File System (IPFS). It's structured and functions similarly to a Unix-style file system

The publish function synchronises your file system with the Fission API and IPFS

Also have been looking at publishing my 'dev-diary' posts on my website, but haven't been able to get the tags to work right yet. I have my own tag plugin for ssb, but indexing the full log takes to long, since patchwork has not ever been using my plugin before. Using the existing tags plugin is also not working and I don't know why. There is something getting mixed up, so that it outputs the wrong set of tags depending on which plugin gets used first. Or somehting like that.

So for now this is written in a text file and being copied to ssb. Copy and pasted text, the most egregious failur for a developer.

Updated the css for jessica's website it's more minimal now. Waitnig to hear for more feedback.

eventual-gram is more or less done. There are still many issues with improving the UI though. I guess those aren't as fun as other things so it is taking me longer to do them.

I still would like to publish the dev-diary entries on the web, however I can't get anywhere with the ssb-tags plugin. It says the structure is like, but none the less i can't find a tag...

{
  // a user id
  '@DOIjef...': {  
    // a tag id
    '%cJEMdje...': {
        // a msg id
        '%x423jsadxj...': 1204594095 // timestamp tag was created
    }
  }
}

Next step is to dig in to the patchwork source code and see how it's done there. Meanwhile copy and paste the content.

Currently have been working on 'ssc', which is like a pub but more boring and cheaper to host. Need to get message validation working on the node side. It's cool to look into the 'low level' source of ssb and re-do things. It strikes me that a lot of code can be removed/simplified.

This part is just rambling.

It's funny working on things again. I notice that somehow things take a long time, but I don't know why. Once you know what to do it should be relatively quick to write the javascript for it… maybe it takes a few days of mulling things over and making revisions of a blog before you realize what you should do though. That's something I've noticed — it just takes a little while to figure out what to do. Also boring things are less fun, so even if you know what to do, you put them off for longer.

#dev-diary

I got a hardware 'key', which i hoped was like basically a public & private key that is tied to the device somehow. It's actually kind of a complex thing & requires using some APIs that are kind of obscure.

Found cypher-net, and old dominic project.

Watched an old video -- 2013 Realtime Conf. git replication -- group the hashes into common prefixes, and then hash each group. That way you can tell if any of the groups contains a change, so it's more efficient to replicate.

Can't get the tags to work on the main ssb network. It returns undefined or something like that. The new plan is to write things in this readme and then parse the markdown and write it to the website, and copy paste to ssb. I guess i could also just get a stream of this feed, which is mostly development logs.

I feel like I haven't gotten too much done this week. I applied for a job. It's at some kind of local dev shop here in bellingham weirldy. So that will be nice if that works out and I can move back to california. Otherwise just kind of poked around with things. Some ssc stuff.

ssc is what i've called the next project btw. it a 'pub', but made with unique code (it's not a regular ssb peer). ssc is like ssb but newer because c comes after b in the alphabet. Its ssb, but run with contemporary things -- put it on netlify, use faunaDB for storage, lambda functions. It's not any less decentralized than the current system with pubs sort of, but it uses more boring stuff -- no webscokets or RPC. The part that might be less decentralized is that it is different code than the 'client' apps. Also have to use a different network b/c it uses different protocols.

todo

• What is a distributed hash table?
• Need to work on ssc -- get replication working

Have been thinking about git/merkle-dags/ssb today. I was wondering does git sign commits? I know that is the premise behind ssb, everything is signed and also a merkle-tree. I found this fantastic article about it: https://withblue.ink/2020/05/17/how-and-why-to-sign-git-commits.html .

So no, by default git does not sign anything, it just creates a merkle-dag, where every commit is a hash containing the previous commit (i think).

I think this train of thought started by thinking about more general ways of doing ssb. The technology is pretty generic after all -- hash functions used in a particular way. It's kind of a shame that it feels so tied to a particular implementation. I guess the database/user-id and replication are kind of coupled at the moment.

Anyway in the article they mention something called gpg, which I think is a general unix way of doing digital signatures (like we have in ssb).

It makes me wonder about having a ssb client where you can choose an identity, like using your gpg credentials, or metamask or something. Digital signatures have gotten more attention because of bitcoins maybe.

Almost done with eventual gram, so that's good i suppose. Have been thinking a lot about ssb running in browsers, and I kind of realized that the only thing that makes ssb a cool radical p2p project™ is that you have all the data on your computer. That's what makes it independent from a centralized server -- there is not one place where the data is. So maybe electron is not so bad. It does what is needed — unlimited local storage with a web-based UI.

If you go the in-browser route of using indexedDB as basically a local cache, you lose independence from servers, and you basically have something like firebase or rxdb.

Also added more photos (postcards) to my website. I noticed that my plugin, ssb-tags, doesn't work the first time you use it, so need to figure that out.

Found some great writing yesterday

• https://www.inkandswitch.com/local-first.html
• https://github.com/automerge/hypermerge
• https://github.com/automerge/automerge
• Verifying strong eventual consistency in distributed systems
• Specification and Complexity of Collaborative Text Editing
• A comprehensive study of Convergent and Commutative Replicated Data Types
• CRDTs and the Quest for Distributed Consistency
• Data Laced with History: Causal Trees & Operational CRDTs
• Project Cambria

Have started off today by digging into the awesome resources created by @RangerMauve on github . Can't remember how I got started on this train of things… I was updating my website nichoth.com. It pulls posts/photos from my ssb client and writes them to an html file using hyperstream. I got the tags plugin to work also. So photos can be indexed based on tags now in the website.

Need to fix things on eventual-gram… when you first run the app it doesn't know your name or avatar.

The difficult thing for all p2p programming seems to be local storage at this point in time. There is indexedDB in the browser, but that is more like a local cache because there is no way to know how much space you will have or when things will be deleted. This got me thinking about out of order messages/partial replication in ssb. It seems that is the only thing missing to let it work in a browser. However I can't say that I understand the terminology being used in the existing writing about it. For example, here, I don't get the use of the term 'merkle tree'. My understanding is that ssb currently does use a merkle tree, which is just the git-like structure of using posts that contain a signed hash of the previous hash (blockchain-ish).

It seems the best option is p2p + a 'pinning' server to make sure things are available (which is like a pub server in ssb).

• https://github.com/ssbc/ssb-server/issues/124 -- partial replication issue
• https://github.com/ssbc/ssb-db/issues/27 -- 'Merkle Tree Logs' issue
• https://github.com/arj03/ssb-browser-demo
• https://github.com/arj03/ssb-browser-core
• https://github.com/arj03/ssb-partial-replication
• https://github.com/ssbc/epidemic-broadcast-trees/
• https://viewer.scuttlebot.io/%25L9m5nHRqpXM4Zkha1ENTk5wNOXQMduve8Hc9%2BF0RLZI%3D.sha256 -- ssb thread about enabling partially subscribable feeds
• https://github.com/dominictarr/tree-exchange

#dev-diary https://github.com/nichoth/eventual-gram-ssb

special bonus edition re: radicle

Read about radicle today. A nice article

From the article:

Radicle Link extends Git with peer-to-peer network discovery. Taking inspiration from Secure Scuttlebutt,

Which is ironic, because I thought peer discovery was the weak point in the ssb system. It depends heavily on pub servers, and maintains a simple list of peer (server) addresses to try to connect to. Which is not to be critical of the way ssb works, it's kind of just an insurmountable obstacle — peer connections over the internet.

What is radicle?

returning to a fully distributed model

It seems that radicle uses a 'local-first' approach with issues/project metadata in addition to the source code (git files).

Ironically I heard about radicle on twitter (the opposite of decentralized) from the person @Kira I think.

I don't think I've done anything substantial lately. I was thinking that eventual-gram is more or less 'done' now. I need to do a couple things so it's more usable then it's time to set it free.

Also read about gemini and agregore browsers. It seems like people are fond of using dat now. I think beaker has built in dat also.

https://github.com/nichoth/eventual-gram-ssb https://github.com/nichoth/earthstar-demo https://github.com/nichoth/ssb-tags

It's a new day. Made some github issues for lingering questions. @cel gave an awesome answer yet again.

Started digging into earthstar also. It's super cool. There is a pub like project for synchronizing data. It reminds me of SSB except they got rid of the append-only quality. The storage of data is the biggest hurdle in my view. Well, storage and synchronizing with other 'peers'. In SSB, for example, we have pubs to help with it. I don't know, maybe you always will need a server to help things. I was thinking just now about using new experimental browsers like beaker. Still need to learn about that.

Was doing some experimenting thinking about what SSB would be like using the most boring things. Which makes you think, what is SSB really? There is

• append only log for feed storage
• a DB with views that create feeds
• public/private keys for identity
• blob storage -- multiblob i think for ssb

The key is that replication works well b/c of the merkle tree quality. The other key is electron -- letting you run a server on your computer is an effective way to do p2p. Everything in the list can be hosted for free more or less on netlify except for blob storage.

Also made a view for tags. that's fine. as simple as possible. It doesn't create any message types in ssb, just creates a view like { tagName: [msgID] }

https://github.com/nichoth/eventual-gram-ssb

Thinking about how replication works now. The protocol guide is pretty helpful. One sbot creates an RPC connection to the other, and calls createHistoryStream for every feed that they follow. I still don't understand how permissions work for the RPC calls however. Maybe it's in the protocol guide too.

This comes from a context of thinking about what a re-implementation of ssb would be like, so that it is more like git & github. Meaning you have a more traditional client & server structure. It's kind of like pubs now. But what would it look like just doing http calls instead of RPC over TCP?

Have been playing with making a flumeView for tags also. The existing tag structure feels kind of weird to me. it is a tag object, which has no data really, it's just an object. Then you publish an about message that names the tag, then you can apply the tag to messages. I'm trying to re-do it so that the name is part of the tag object (you can't re-name tags in this structure). So you have a flumeview that is a map of tag names that points at a list of messages. Updated the field-guide with some stuff too.

{
    name: [msgID]
}

https://github.com/nichoth/eventual-gram-ssb/blob/5d106ef167d9b21453f0143f3aaee8076d8a5c3d/src/subscribe.js#L126

Its more complex that I expected. It seems that a tag is it's own entity in the DB, not even having a name property embedded in it. The tag object then has to be named with an about message. It seems more convoluted than i would expect, but maybe there is a reason for it. I would expect name to be just a property on a tag, then if you want to change the name you would create a new tag, and you would need to publish a message that re-links all the posts that the tag references.

https://github.com/nichoth/eventual-gram-ssb/blob/main/hashtag-test.js

tags

Tags are pretty important b/c they are the main way of categorizing messages. I think in patchwork hashtags are called 'channels'. I looked around in the patchwork source to see how it's done, but didn't figure it out. It seems that tags are their own type of message that is published to the log and replicated with everything else. I've been looking at using scuttle-tag thus far. It's not clear to me what the signature to create a tag is though. The docs feel slightly incomplete here. For example, where is the name (the text) of the tag saved? And is this a ssb plugin too? I'm not sure based on the signature.

Actually, it looks like the tag name is published as a separate about message. So creating a tag with a name is a 2-step process — first create the tag, then name it (then apply it to the message being written as the overall last step).

scuttle-tag has a getSuggestions method. Need to try that so you're not constantly creating new tags with the same name.

Plague life is kind of a bummer. That's pretty wild that just a few months ago we took it for granted that we could just meet with anyone and not wear masks.

https://github.com/nichoth/

Just got back from camping. Haven't done much devloping I think since the last diary entry. Not sure if I wrote about it last time, but i think i finished the website for now. It's nice because I used ssb as a content source for part of it. ssb really is a nice general use of the merkle dag. like git but for arbitrary data, and with an easily replicable identity structure built in. Have been thinking it would be cool to make the ssb photo app more real, with a built-in preliminary pub that I host, and a web interface (kind of like how github is a web interface for git).

Now i have to start actually looking for a job now that the website is more or less done. Preferably something in europe or south america or something. Anywhere more stable than here at least.

The dev-diary does feel more trivial now that I'm living through an apocalypse. During the drive back to civilization it was a lot of talk about how we need to find a new country to live in b/c this one has gone bad. You know, the usual. I saw @Rabble mentioned on twitter moving to I think Uruguay? Ever since then I've had that idea in the back of my mind.

I recently lost most of my memories, so it's funny when one resurfaces. During the drive I was thinking about ssb, and this story @Dominic told about cryptography came back — the public/private keys being like the story cinderella . Aside from that there is the anecdote about just how new of a field cryptography is. The merkle-dag & public/private keys (blockchain) seem like an obvious application in retrospect, but it's just being developed right now.

#dev-diary

https://github.com/nichoth/nichoth https://github.com/nichoth/ssb-web

Saw a nice episode of speakeasy js , in which rod vagg discusses some basics of content addressed blobs.

Also, have finished my website, which uses ssb as a content source, and renders the html with hyperstream.

I was thinking while making this that it would be good to query by both userID and and post type, which interestingly I don't see a way to do with the existing API. But maybe it's not necessary. So far doing the query by userID, and then using pull.filter to filter the results has been adequate. It's kind of an interesting rabbit hole, how to do efficient queries when you already have 2 materialized views. I think your query would be items that are in both the userID results and the .type results.

Another question is how to do hash tags in ssb. I discovered the plugin scuttle-tag , which seems appropriate. Also discovered ssb-conn, which i think phases out the gossip plugin.

https://github.com/nichoth/nichoth https://github.com/nichoth/ssb-web

In the process of cleaning up/adding content to my "website". This is relevant to life b/c I'm going to look for a job soon i guess. Related to the website I started dipping my toe into using ssb as the content source for a webpage. That's what ssb-web is. I started the website project using eleventy, but then decided that it would be simpler with hyperstream, an old substack module. It gets more complex when you have a situation with nested templates, in this case when you have a page of posts from ssb (a template), and each post is created from a template. I guess it's working now. It just feels kind of spaghetti-like.

Haven't done much with eventual-gram lately. It feel like it's almost "done", but just needs some better UI with pubs and following people.

https://github.com/nichoth/eventual-gram-ssb #dev-diary

I implemented the function that @cel posted and it works great. I started by getting all contact messages and finding the ones i needed, but that took way too long, like 10 seconds. Then I thought I could just as well make a materialized DB view of it ( a flume view?) , but the way in cel's example is totally fast enough. I hadn't seen the sbot.links method previously.

I started doing this originally b/c i wanted a place to share photos, but I couldn't bring myself to use instagram. I wanted something like ssb as far as the feeling of using it, but more for photos. Anyway, was reading this article about climate change, and realized this is the last generation to experience life in a world without a destructive climate. I had accumulated quite a lot of photos, but things have already changed so much in the last year — covid, fires — they look like they are from a completely different time. Up until very recently this country I guess has been kind of riding the wave of baby boomer prosperity, and you can kind of see it in the photos. They are mostly of people doing weird things or pictures of squalor, but they just aren't great to look at now. There is so much trouble all around… it feels suddenly dated I suppose is the short way of saying it.

hmm i feel like i had a clear intention when i started writing this, but that's close enough i guess

https://github.com/nichoth/eventual-gram-ssb

Something funny happened today. While working on the new version of the app, my test posts from the previous version showed up. It not bad, just kind of weird and cool to see your old data. I guess it was < 2 hops away in the pubs.

That's what I'm working on now, pubs & following people. The tricky part seems to be getting a list of everyone that I'm already following so I can adapt the UI accordingly. There doesn't seems to be a method in ssb that does exactly that. There is friends stream , but this returns too much info — it is a list of who you and all your follows are following.

Or there is gossip.peers which is not ideal b/c I think it returns a list of everyone within 2 hops (not just people you are directly following). It returns a list with hundreds of peers in it, not just the people i have followed, which is like 2.

It's a little weird b/c it seems like a fundamental thing to know — who you are & aren't directly following. Maybe I'll need to just get all the contact messages and reduce them in the app code.

https://github.com/nichoth/eventual-gram-ssb

A cool thing about devloping is that you can loose hours just sitting and not doing much, but you're reading about various things on the internet. It's no wonder the creator of ssb lives on a boat — you have so much more time when you don't have to pay rent. While i don't recommend the whole severe head injury thing in general, living at my parent's house is great for options with how you use time.

Just now for example I was reading about the question of how replication works in ssb. I understand at a high level the system of following & public/private keys, but it must come down to a URL & protocol for a peer at some point. I think it can use various protocols in addition to tcp.

Looking at the code here https://github.com/ssbc/ssb-invite/blob/master/index.js#L195 , it appears that all that info of host & protocol is embedded in the invite code used by pubs . It's still not clear however where the pub's address info is recorded in the DB. It must be written down somewhere.

The "app" (the instagramish thing) feels like it's in an ok state for some reason. it doesn't do much of course. I'm wondering if this feeling like it's semi-done has more to do with my brain being in an ok state than the app being ok. It's stable at least now, no disappearing posts anymore. Next is make a way to add peers and pubs. Important for a "social" app, seeing peers.

Next step I think is fixing up my "website", which not so surprisingly leads me to want to make an ssb-backed website generator/CMS tool. Related to the website task is that I am going to start looking for a job i guess.

I was thinking I should make a pub for my app, which made me think about what it would take to make one for free. This means you would need to use lambda functions (via netlify) and re-write some of ssb so that it uses finite processes instead of a daemon-like process that is always on. The part that costs money then is the storage. Netlify has a nice free-tier of fauna DB, but of course you have to pay once it exceeds the free part. This is a long way of saying that it would be cool to make an ssb that uses finite processes, which is something else on my list now.

I still need to learn what a decentralized hash table is btw. There are so many places where things could be split up still. There is the DB layer, the networking part, and the ID part. I think it's mostly all in one with ssb. I was looking at the prior library scuttlebutt actually and it seems to be more in line with what i would like ssb to be -- just a database layer, not networking. An api that handles the DB and gives a stream interface for replication/networking.

In real life, we're mostly talking about how it's kind of interesting that the country is literally disintegrating into chaos right now. Talking about countries we would like to move to, what to do, etc. Erin has mentioned the tensions between getting a "real" job and just kind of hanging out and growing plants and mushrooms. Not sure which would be more stable at this point. It's funny that you really can feel the difference between life now and a couple years ago. Part of it is aging, which is normal, but also the world is new. We are now unstable.

https://github.com/nichoth/whammy https://square--whammy.netlify.app/

Have switched back to doing ssb-ish stuff -- https://github.com/nichoth/eventual-gram-ssb . While developing, I had a small DB seeded with data, but then I enabled the plugin ssb-db, and after that the database was totally borked, and the file it was looking for was missing. So that's a bummer.

Have started looking at 3box and textile, though there is still some weirdness with them that requires some context specific knowledge. Like you have to make calls to an ethereum library in order to use 3box, and that's fine except that i don't know what ethereum is or how to get it in the browser. It's frustrating. You have to go to great lengths to do simple things.

#dev-diary

Haven't done any work on the ssb/photo sharing app ( https://github.com/nichoth/eventual ) for a while. I've been distracted by making a store that sells vhs tapes on the internet. It's called whammy analog and it's kind of fun to make because the cool thing about it is that I'm sticking to a buget of $0. We're using netlify for hosting and square for the payment processor & inventory CMS. Originally I was trying to making it all more 'from scratch' by using netlify & faunaDB as the CMS/inventory tracker, with stripe as a payment processor. Then i found out that square has already built all those things for me. The one tricky part now is finding out how to apply a shipping cost to an order. There doesn't seem to be any docs about it weirdly.

Hmm I suppose that's all I've been doing lately. I feel like there should be more to talk about for some reason. On the horizon there is

• ssb photos app
• my 'personal' website
• a 'graph' db type of knowledge hoarding app

#dev-diary

Reading here https://ssbc.github.io/scuttlebutt-protocol-guide/#blobs

blobs.get asks a peer to send you the contents of a blob.

All blobs that you have on your machine are hosted at localhost/<blobURL>. See ssb-serve-blobs . I guess you need to call blobs.get first, then after that any blob is served on localhost.

Also saw this dat photots app. I have no idea how it works though. there is not even a package.json.

https://github.com/nichoth/eventual #dev-diary

Confused about blobs in ssb. After doing some reading, it seemed as though they are replicated "automatically", meaning any blob in the mentions array is downloaded, and it's up to the users to for example not follow someone who shares an excessive amount of data. But looking at the docs for sbot.blobs confuses me, in particular the want method https://scuttlebot.io/apis/scuttlebot/blobs.html .

Also have decided to get tests going on the local machine so that it runs 2 sbot instances that can replicate with each other. It's going to be difficult i imagine, a day time activity. Testing in a 'live' scenario with another person is fun when it works, but is kind of time consuming now.

https://github.com/nichoth/eventual

#dev-diary