I installed the Nov 17 update while logged off; 22 days of data lost?

šŸžBug Reports
#1

To get your bug fixed faster please try to answer the following questions:

  1. Which browser, operating system and versions of those are you using?

I use ungoogled chromium which seems to get frequent updates. Iā€™m on Chrome 96.0.4664.45 right now. I also have to use this non-standard extension upgrade process: GitHub - NeverDecaf/chromium-web-store: Allows adding extensions from chrome web store on ungoogled-chromium. Also adds semi-automatic extension updating. but that seems to work well too.

  1. Which version of Memex are you running?

Nov 17 Update

  1. What are the steps you took that lead to the bug?

I didnā€™t use my Mac for maybe 5 or 6 days, which is unusual. When I started it up today, several of my extensions needed updating including Memex. This may have been an unusual case; I took an update when my auth cookie w/memex was timed out? It looks like that is what happened.

Problem is, nothing notified me that I was not logged on to memex. I didnā€™t notice anything was wrong until I tried to search for something and only had 11 items in my collections. From today I guess. Dā€™OH! Then I noticed I was logged out, and was relieved because I figured either my backup server or the backend sync would save me.

Double Dā€™OH! I watched it sync 0 items down from my account (how is that possible?) and 11 items up. When sync came out, I took that to mean that in addition to my local backup server, my data was encrypted and stored by memex as well. Is that not correct? Then I visited my backup screen which reports:

image
image831Ɨ297 16.2 KB

I donā€™t know how many Dā€™OH!'s thatā€™s worth, but a lot. So unless we can reconnect things with the backend somehow, looks like Iā€™ve lost a good chunk of data.

I was on such a roll! Iā€™ve had an account with Memex for a long time now. I would have lost data a few times in the early days, but I wrote my own scripts to merge my split brain problems back together.

Do I owe you a yearly payment or something?

Edit: in summary, if the problem was that I was not logged on for a long time (perhaps even before I stopped using my Mac), AND my backup server had stopped runningā€¦ then we need some kind of nag that is a little more aggressive. Come to think of it, I donā€™t think Iā€™ve ever seen memex nag me to enable backups or login?

#2

Oh, also, I will wait on restoring from backup until I hear back. Just in case there is a complete rescue possible :crossed_fingers:, but I would make things harder by restoring what I have.

#3

Hey @GeekyE

First of all, so sorry that you (potentially) lost your data.
Seems like you indeed have an unusual setup that could have led to this, but I really want to understand how we can make things more failsafe even if that was the reason, and especially if not.

Letā€™s hop on a call, difficult for me to investigate things remotely:

Speak soon!

#4

Just to close this up, wanted to thank @BlackForestBoi for helping me restore my complete timeline of data! Thank you for your time.

On the plus side, as far as I can tell, I have restored 100% of the data into one profile, and it is all backed up on the Memex cloud as well as my local machine. :tada: The reason all of my data was recoverable was that even if the extension refuses to sync down, it will still sync up from a mostly empty profile into your full collection on the server side. So my first clean restore included the very latest bookmarks I had made from my disjoint profile.

So there are a couple take-aways:

  1. Basically what happened was another split-brain, just like I struggled with when I first started using the product. Memex uses a log-replay model for the data store. I keep managing to get multiple histories going, and just like git pull, memex can get in a state where it canā€™t (or wonā€™t even try) to merge the histories together. Things are a lot better now, with the sync feature, but the problem is still there. It should be easy to repro it: have chrome setup with a memex collection, syncā€™d and authenticated. Quit chrome. Delete Chrome/{Profile:Default}/IndexedDB/{memex-ext-id}{.leveldb,.blob} (if you have .blob, my original profile did). Do not remove the other traces of the extension in the profile. Launch chrome. At this point, it should allow you to login from within the extension (you are no longer authenticated). But you can also make new bookmarks. Make at least one. Then authenticate. It should show ā€œ0 items to downloadā€ during the sync, no matter how many you have on the server side. However, it will push your new bookmark up to the collection. The correct way to recover (if you get in this half-initialized state) is to uninstall the extension, and reinstall it. It will prompt you to login as part of the extension install, and your full collection will sync down from the server. This seems like a fixable problem, it just needs to merge entries. There must be code to do that, because sync works. A 2nd option would be to prevent users from starting new histories if they are known to have an account on the server side. Maybe that would be safer, to prevent merging unrelated profiles together. Either login to your existing memex, or click ā€œcreate a new timelineā€ or some-such.
  1. Iā€™m not sure what the extension can do to get the userā€™s attention once youā€™ve gotten into this risky state. Can the extension bounce around off the toolbar like it was an app in the macOS dock? That would work! Yeah I know that isnā€™t going to happen. But it needs a nag of some kind.

PHEW ok, thanks for keeping this product maturing!

Ethan

1 Like