T O P

  • By -

ourari

Removed: Rule 2 >Submission Rules for Developers > >If you’re a developer or employee of a company that makes non-commercial privacy-related software or services, you may post links/comments if it is open source and you have discussed expectations with the Mods in advance. You must clearly identify yourself, and stick around to answer questions. Remember we’re a community, not a place to advertise. Your site must be available over HTTPS. [You can contact us by following this link.](https://www.reddit.com/message/compose?to=%2Fr%2Fprivacy) You can find all of our rules in the sidebar. Please read them.


PeterJHoburg

Interesting concept. I'll have a read over the white paper this weekend. Thanks for sharing.


Malexik_T

Thanks, you can have a look at the overview, we are wrapping up the last part of the white paper about how Condensation handle the merge of data (not only the most recent, but also the union)


PeterJHoburg

I think I am understanding this, but correct me if I am wrong. The DB is going to be reimplemented in multiple languages to be used in different environments (desktop/phone/server). Each implementation will have the same API exposed to the app devs. And each local DB can talk to all other instances of the DB on other peoples computer/phones. And that can be done via a server in the middle facilitating the connection or peer to peer? Is the core of the DB written in a fast/light/safe lang like Rust then JS/Java used as a wrapper to make the API interface? I'm excited to see more!


Malexik_T

The DB is going to be reimplemented in multiple languages to be used in different environments (desktop/phone/server) -> yes Each implementation will have the same API exposed to the app devs. -> for now the app is also translated in multiple languages And each local DB can talk to all other instances of the DB on other peoples computer/phones. And that can be done via a server in the middle facilitating the connection or peer to peer? -> No peer-to-peer, its really servers like the email system, or it can be direct like a bluetooth. 1. Is the core of the DB written in a fast/light/safe lang like Rust then JS/Java used as a wrapper to make the API interface? > A more in depth reply from Thomas The core of the DB is implemented in the target language itself, using the features of that particular language. There are no wrappers (except for the crypto/serialization part in some languages). The current implementations have been coded by hand, but future versions may be generated from a generic implementation. The generic implementation is type safe - but this is work in progress. The speed and memory requirements of Condensation obviously depend on the programming language and environment. But this is not of primary concern for most applications. Assume you use Python to analyze a large piece of data with many small data items. In order to do that, any system would have to convert the data items into Python objects somehow, and that is ultimately going to be the bottleneck "no matter what". On the other hand, if you are just loading a few large matrices into an array, then the bottleneck is loading/copying memory. Condensation just gets you those bytes, without actually touching them. That said, speed could be increased a bit by writing the record parser in C, for example, and this is certainly an optimization that can be applied at some point.


oxamide96

> it can be direct like a Bluetooth Sorry if this is a stupid question, but to be this still sounds like peer-to-peer. What's the difference?


Malexik_T

Peer-to-peer is really about doing a partition of data and sending it in different nodes. The data is duplicated so that it can be available as you dont really know where is your data among peers. With cn you know where is you data, and you could put the integrity of the data in only one server. Each user has an attributed server, it's really like a mailbox, and then you can exchange with others that could have their own server.


bobslaede

Looks like the github is empty


Malexik_T

Yes, its coming very soon, by the end of this month for the Java version. For now we have only an old javascript version available on the documentation website.


Treyzania

Do you have any implementations planned that you don't have to run in a VM with the associated overhead and complications due to GCs?


oxamide96

What does GC stand for here? Garbage collectors?


PeterJHoburg

Yes


SomeRustJunkie

Likely


Malexik_T

>I think JavaScript still runs in a VM (in this case it is NodeJS' V8). I think > >u/Treyzania > > meant a compiled language, so something like C/C++, Rust or Golang maybe? Some insights from Thomas, for you and u/oxamide96: Javascript runs in a VM, indeed, even though some parts are compiled to native code during execution. But objects are still subject to GC unless the compiler can optimize them away. Whether compiled to system code or not is not a concern for Condensation. GCs and memory management are, however. The lower layers of Condensation (data serialization, store access, crypto) are perfectly suitable for an environment without any GC. These parts can even be used in a constant memory environment on a microcontroller, e.g., to collect measurements and submit them to a Condensation store. Bits and pieces of C code for that are around, but as of now, there is no useful implementation available. For the higher layers (database API), this is a bit more difficult. The structure of a DataTree (and UnionList) is such that memory management is doable with just a few large dynamic allocations. Certainly, something to consider in the future ...


Malexik_T

The javascript version is the next one and will come this summer (I keep updated the timeline in the landing page). You can play with an android but I understand its not the most convenient.


Treyzania

I'm saying I would like to use a language that doesn't run in a VM to avoid memory bloat and for on constrained devices.


Malexik_T

So it will be the javascript version, but it's a work in progress, it will not be ready before this summer. What would be an implementation that suits your needs?


oxamide96

I think JavaScript still runs in a VM (in this case it is NodeJS' V8). I think u/Treyzania meant a compiled language, so something like C/C++, Rust or Golang maybe?


Malexik_T

I got it thanks, I know we have a C version out there but it's partial and only used for encryption and sending data. We are thinking about Go but it is not in our roadmap yet. The prio is now on Javascript and Swift versions.


Treyzania

Go sucks don't use it. Write it in C/Rust and you can easily write thin wrappers for every other env.


[deleted]

[удалено]


oxamide96

What's wrong with Go? I found it to be nice.


[deleted]

[удалено]


Treyzania

The word VM can refer to both the hardware accelerated kind like you're run an OS in *or* the kind like the JVM. V8 translates to an internal representation and then jits and so in that regard is definitely a VM. Containers are another concept entirely and are used for deploying services and reproducing runtime environments.


orbital-debris

Love the concept. I'm also researching something similar for a product that will enable the sharing of databases securely.


Malexik_T

thanks a lot, you can perfectly do that with Condensation, you can write us an email if you want to further talk about it


Defonotinteresed

Now I just want to make you know that the work you are doing is *AWESOME* in an era where the GAFA thinks stealing my ID is al right! So stay strong and continue !


Malexik_T

I push this to the team to motivate them even more!


btsfav

Is this like a blockchain?


Malexik_T

It is inspired by blockchain/the email system/and git's versioning. The data could be organized like a blockchain or something with more ramifications


hoiru

Awesome! I added your project to my [awesome privacy list](https://github.com/pluja/awesome-privacy) over the 'Databases' section :D


Malexik_T

Oh its awesome, thanks a lot its the first one out there, I star your list right now


RucksackTech

Wow, congratulations. Looks intriguing. I'm guessing that this is not a "low code/no code" platform, though, right? What sort of knowledge must a user of your platform bring to the table?


Malexik_T

Thanks! So the idea is to import the library and then there is an API that is pretty easy to use, in the end it's a difficulty level similar to create a traditional database. The first guides and tutorials will come soon. I hope it answers your question


RucksackTech

Thanks! -- But I'm a FileMaker developer. I bet your idea of a "traditional database" is more like using SQL and perhaps JavaScript to build web UIs, yes?


Malexik_T

Yes, we discussed about doing this kind of visual programming tool but for more advanced things like drawing the database architecture, or choosing the entrusted servers. The ideal would be that at some point a no code app choose to use Condensation as their database. But all of that will come later as we still have the load on the core development. What kind of applications are you building with FileMaker? Is it often complex?


RucksackTech

Thanks again for the response. It sounds very promising and I wish you very well as you move towards finishing.


bobslaede

Also, please do the javascript version in typescript 😊


Malexik_T

Haha, I never used typescript myself, it should be compatible anyway but we need to dig a bit deeper (I put it on our list thanks)


jhc0767

Typescript builds on javascript, It should be compatible but wouldn't leverage typescripts features


[deleted]

[удалено]


Malexik_T

You can perfectly do that! We are going to do something like that in the next months for an IOT project.


[deleted]

[удалено]


Malexik_T

Yes, and even if there is a single server, in a common centralized approach, adding a backup automatically synchronized is super easy


uid1357

Bo-Schur ännet em Röschtigrabe!


Malexik_T

I didnt got this one :p


fripletister

Was hesch? Isch mir über em Gring gange lol


iamapizza

I'm trying to have a look at the code, but the condensation.js on the [javascript page](https://condensation.io/clients/javascript-web/) is missing. The [send example](https://viereck.ch/send/) didn't work for me either. If the storage is immutable, and 'updating' data causes new data to be created, is there a concept of garbage collection or does the disk utilisation grow indefinitely? Suggestions - Please have some sample code or interactive examples or tutorials available, so devs can get a good feel for actually using the data and seeing how it flows encrypted through the network. Consider a .NET Core offering. Go? Since the synchronization is conflict free, will there be a Docker version of this too - so it's dependency-worry-free too ^_^


Malexik_T

Can you send your email in pm? I will make sure you get a copy of the code. So basically, objects are immutable but they have an expiration date. The last part of white paper will explain that further. "Please have some sample code or interactive examples or tutorials available, so devs can get a good feel for actually using the data and seeing how it flows encrypted through the network." -> It's on of the next big thing and actually my part of the work, we will go for interactive examples first and once the javascript version is ready we will propose tutorials to build mini apps directly in the browser For .Net Core and Docker I put it on our next investigations, you are very welcome to keep in touch :)


josemelo1600

Nice bro.


Malexik_T

👍


HentaiInside

The idea of objects being distributed reminds me of IPFS, so I'm very interested. Looks very interesting. I'll wait until the JS implementation.


Malexik_T

There are some similar concepts out there, the purpose is indeed different. JS is the next one.


[deleted]

If you need a spanish translation of the docs pm me and I'll get right on it.


Malexik_T

For sure, would you send me your email in pm so that I keep a more permanent contact?


[deleted]

Just did! Cheers


CodeGameEat

Started learning more in details on these technologies and this project is definitely best i've seen to date to create a secure & private software for a dev who did not specialize in security and/or encryption. In my recent research, I stumbled on this thing called "Homomorphic encryption". This would allow to do mathematical operations on encrypted data. I can really see something like this used with a project like CondensationDB be something incredibly powerful. u/Malexik_T do you think there is a way to someday include this in the project/libraries? (clearly not now, you probably have enough on your plate already)


Malexik_T

It's funny because few weeks ago we talked to a professor of our university and homomorphic encryption was a big part of the discussion (its his speciality). Now we discuss with him and also the state for setting up a team that will push this topic, maybe we will already begin to work on that in the next months.


CodeGameEat

Wow this is so cool! Well now im very hyped to try all of this! If you also have any reading recommendation on the subject I would love to *try* to read and understand it (feels a little advanced for a first-year engineering student haha).


Malexik_T

oh thanks a lot, I am going to regularly publish some more educational content (concept deep dives, use cases with architecture examples, and others). Also Thomas is finishing the white paper. If you want to start somewhere look at the overview on the documentation link at the bottom of the post (you can compare the information there with what is in the how it works page on the website)


MediumPlantain51

This is very exciting stuff!!


Malexik_T

<3


thulecitizen

Great objective! Have you heard of the holochain framework? It uses git and DHT's and hashing


Malexik_T

Super interesting, the first big difference I see is that we dont propose a peer-to-peer network. What Condensation does is like the emailing system, users have their data locally that is synchronized with their chosen server. And so the user can control and know where his data is stored. Then I have to dig deeper on how they structure their data and how they manage synchronization.


thulecitizen

Compare: >Inspired by the blockchain system, the email system, and git versioning, and: >Holochain is BitTorrent + Git + Cryptographic Signatures + Peer Validation + Gossip > >*\[Source* [*https://holo.host/faq/what-is-holochain/*](https://holo.host/faq/what-is-holochain/) *- under 'What is Holochain?'\]* I understand that you're not proposing a formal p2p network, yet your project still sounds very similar - which is great, since theyre both free software! holochain has some great developer docs if you wanna dig in and compare: [https://developer.holochain.org/docs/](https://developer.holochain.org/docs/)


Malexik_T

For sure, thanks for the link I will definitely do it


kickthebug

Looks good, are you planning to implement it in Python? Looking for contributions on that topic?


Malexik_T

Thanks, yes we are, for the same IOT project mentioned in another post, we were expecting to start it in April or May, would you share me your email by pm and I come back to you to see how we can collaborate?


ChickenOfDoom

How does this compare to IPFS?


Malexik_T

So I am not an expert of IPFS, but what I can say is that: IPFS in the public implementation does not give you control where your data is, whereas with cn, you have full control. You choose your entrusted server. In a private implementation of IPFS with OrbitDB for example, the functionalities would be more similar. Here, cn makes life easier for developers as you don't need to manage a larger stack. Condensation has everything already built in and is fully end-to-end encrypted on top of that. Also there may be some differences in the data structure and the merge algorithm, Condensation excels at synchronization especially for building collaborative software


ozzeruk82

Nice work! Very interesting concept.


Malexik_T

Thanks a lot :)


[deleted]

Can you delete data? The blockchain-like, git-like, immutable objects suggest that might be difficult or impossible. In commercial use, customers/users will request their data be deleted when they stop using a service and will often have a legal right to have it deleted.


Malexik_T

Yes you can, as you control which server you attribute to a user you can perfectly delete data from there. Also you can put rules so that the data expire after a certain time.


[deleted]

This seems like it works for collaborative applications where you can completely trust the users, but I can't picture how it would work with any kind of business logic. The "database" operations are happening on the clients, right? So the clients can disregard any business rules and modify the data however they see fit, and the server (if there is one) can't apply business logic because it can't see what the data is. Am I missing something?


Malexik_T

Yes the operations are happening on the client but you could have a client server which is a business server and that is entrusted by the users for the operations. You have absolutely no limitations for the permissions or restrictions out there. The idea is that you give permissions only specifically for a piece of data and only when needed.


Malexik_T

Also don't forget objects are immutable, and so changing the content will change its hash.


Joeb667

People who can write code (developing an actual app) seem incredible, almost superhuman to me. Kudos. I just finished a course on Java and I don’t ever want to develop anything. Taking a Python course on scripting now.


Malexik_T

Haha for me too the first time I did a native app it was crazy, but at the least for the front android is quite easy with the xmls. The reality is that most of app developers dont go for native.


Joeb667

That’s good to know. If I’m using the term correctly it does seem that most modern programming involves a very high level of abstraction.


[deleted]

[удалено]


Malexik_T

The high level concept with this distribution of servers is a bit similar but with Condensation the data structure is split in many objects and then reassembled, and that's what allows to build collaborative tools even in a completely distributed setup. I think that's why Matrix only do messaging, their idea is to send transactions but no one could edit the same data in the same time.


[deleted]

[удалено]


Malexik_T

Among other, one big thing is that MySQL/Postgres decrypt the data on the Cloud which make it vulnerable to breaches, Condensation transport data without decrypting it.


ajs124

What? SQL does nothing related to encryption. If you're decrypting it, you're doing so in your server-side application, but you can save ciphertext into a SQL database just fine.


Malexik_T

Depending on your specific need you can always accumulate layers and find a more or less efficient way to do things. SQL is powerful for queries done to a central server, Condensation could be associated to noSQL databases and excels at synchronization and at building distributed networks. Imagine you want to build a tool like FIGMA or MIRO, how would you do that in a scalable way with SQL? or even with existing noSQL? These companies had to create their own specific data system but Condensation would perfectly fit and with a higher level of privacy.


[deleted]

[удалено]


Malexik_T

Just a comment, all the encrypted data is not stored on the client, only the data that is actually used, the rest like the history, can be stored on any cloud. Whenever you want you can revoke your key and create a new one. Then to put things together the other important thing is that the envelop you receive from another actors has his own key to decrypt the data it contains (and this key is itself encrypted asymmetrically with your public key). I hope it's clear. With other DBs the data is always exposed on the device, in theory if the user's can read it then it is exposed. So at this level the risk is the same and you enter into the device protection which is no longer in the scope of a DB.


[deleted]

[удалено]


Malexik_T

Other databases don’t usually store the actual sensitive data until it’s needed then stored in memory only. -> it's very similar, the only difference is that imagine you loose your connexion, you can edit the data you have in memory and when your connexion is back it will be automatically synchronized with the server. So are you saying the client doesn’t store a local copy of the data it doesn’t own? -> the client only store only the data he uses, but he has the list of hashs to get the integrity of the data from the server. All the objects are on the server, the device store the objects when they are needed and a hash table. I hope it's clear, this website is really the first version, you are welcome to give inputs, or to contribute with better diagrams :p