Hello all,

I'm currently looking into the "places.sqlite" database (FF v52) at the
"moz_places" table, and am noticing a lot of old entries* that are "hidden",
but have but have a "foreign" count of Zero.

* some of them to the, now gone, "docs.microsoft.com" domain.

1) Is there any reason for them to be "hidden" ?

2) As they have a "foreign" count of Zero can I just delete them ?

Regards,
Rudy Wieser

On 16/03/2024 11:08, R.Wieser wrote:
> Hello all,
>
> I'm currently looking into the "places.sqlite" database (FF v52) at the
> "moz_places" table, and am noticing a lot of old entries* that are "hidden",
> but have but have a "foreign" count of Zero.
>
> * some of them to the, now gone, "docs.microsoft.com" domain.
>
> 1) Is there any reason for them to be "hidden" ?

Spying?

> 2) As they have a "foreign" count of Zero can I just delete them ?

Sure. Make a copy of the profile first to revert back to its original state.

With Firefox, the only thing that needs to be backed-up is Bookmarks.
The rest can be recreated as and when needed such as stored passwords
and history. These days people don't save passwords in browsers and
history of visited sites is not important. Perhaps old Add-ons might be
a problem to download but people must have a contingency plan anyway.

Mugomba,

>> 1) Is there any reason for them to be "hidden" ?
>
> Spying?

I'm looking for an reason coming from knowledge, not fantasy.

>> 2) As they have a "foreign" count of Zero can I just delete
>> them ?
>
> Sure. Make a copy of the profile first to revert back to its
> original state.

In other words, you have no idea.

Regards,
Rudy Wieser

On 3/16/2024 6:08 AM, R.Wieser wrote:
> I'm currently looking into the "places.sqlite" database (FF v52) at the
> "moz_places" table, and am noticing a lot of old entries* that are "hidden",
> but have but have a "foreign" count of Zero.
>
> * some of them to the, now gone, "docs.microsoft.com" domain.
>
> 1) Is there any reason for them to be "hidden" ?
>
> 2) As they have a "foreign" count of Zero can I just delete them ?

When the hidden field in moz_places has a value of 1, that indicates the
URL was not navigated to directly by the user. The URL was probably
embedded in the page. Hidden sites do not display in the History panel.
However, if you were to change the value in moz_places from 1 to 0, then
these sites would display in the History panel.

You said you're currently looking at the places.sqlite database. Are you
familiar with constructing select queries for editing such databases?
Even if you are, Mugomba is correct that you should back up your profile
(or at least places.sqlite) first. It's easy to corrupt a database if
you don't know what you are doing. Do any editing with Firefox closed.

On 3/17/24 06:41 AM, gym wrote:
> On 3/16/2024 6:08 AM, R.Wieser wrote:
>> I'm currently looking into the "places.sqlite" database (FF v52) at the
>> "moz_places" table, and am noticing a lot of old entries* that are "hidden",
>> but have but have a "foreign" count of Zero.
>>
>> * some of them to the, now gone, "docs.microsoft.com" domain.
>>
>> 1) Is there any reason for them to be "hidden" ?
>>
>> 2) As they have a "foreign" count of Zero can I just delete them ?
>
> When the hidden field in moz_places has a value of 1, that indicates the URL was not navigated to
> directly by the user. The URL was probably embedded in the page. Hidden sites do not display in the
> History panel. However, if you were to change the value in moz_places from 1 to 0, then these sites
> would display in the History panel.
>
> You said you're currently looking at the places.sqlite database. Are you familiar with constructing
> select queries for editing such databases? Even if you are, Mugomba is correct that you should back
> up your profile (or at least places.sqlite) first. It's easy to corrupt a database if you don't know
> what you are doing. Do any editing with Firefox closed.
It's been a few good years, but I used to program in sql, not in sql-light but from what I remember,
a full blown sql server specifically setup to support the backend of our accounting database.

The one thing I remember (probably the only thing I remember) that pertains to this post is that
databases can be heavily linked between files. There can be tables linked to tables linked to
tables. This is what a true structured database is all about. Editing one file makes others change
just by the nature of it all.

Of course that's in a true relational database. I seriously doubt Firefox and/or sql-lite are being
used that way, probably just a convenient way to store and query data. sql is efficient if nothing
else for doing queries, takes the overhead off of Firefox.
--
Linux Mint 21.3 Cinnamon 6.0.4
Al

gym,

>> 1) Is there any reason for them to be "hidden" ?

> When the hidden field in moz_places has a value of 1, that indicates the
> URL was not navigated to directly by the user. The URL was probably
> embedded in the page.

I'm sorry, but I don't understand that. Apart from a few bookmarked ones
most of the URLs I visit are embedded in one-or-another webpage (mostly
search-engine result pages).

Or, said otherwise, what does "navigated to directly" mean ?

> You said you're currently looking at the places.sqlite database. Are you
> familiar with constructing select queries for editing such databases?

Nope. I'm a novice at best in that regard. But yes, I've written a few
queries and updates.

> Even if you are, Mugomba is correct that you should back up your profile
> (or at least places.sqlite) first.

While the advice to backup something before making changes is generally a
good one, that "Sure," made a mockery of it.

Besides, the bookmarks are automatically backed up (bookmarks-*.jsonlz4)
and, after the database is deleted, can be imported into the new, empty one
FF creates on next startup.

> It's easy to corrupt a database if you don't know what you are doing.

:-) Which is exactly why I asked my questions.

I can *ofcourse* just throw away any record in any table in any database.
The question is if its a good idea - and if not what, in the described case,
I should be aware of.

Did you know that the structure of FF related databases is hard, if at all,
to find ? :-(

Regards,
Rudy Wieser

Al,

> There can be tables linked to tables linked to tables.

Thats pretty-much what my question, and mentioning of that "foreign_count",
is about. If someone knows of such a dependancy and if its relevant.

> Of course that's in a true relational database. I seriously doubt Firefox
> and/or sql-lite are being used that way

Than the following might suprise you :

Some time ago I found an ancient PDF describing the places.sqlite database
schema (FF v3.5, dated 2009), and it shows several tables depending on
others.

https://wiki.mozilla.org/File:Places.sqlite.schema.pdf (the top one)

Alas, I've not been able to find anything more recent (and in my case
"recent" is pertaining to FF v52).

Regards,
Rudy Wieser

"R.Wieser" <address@is.invalid> wrote

| > When the hidden field in moz_places has a value of 1, that indicates the
| > URL was not navigated to directly by the user. The URL was probably
| > embedded in the page.
| | I'm sorry, but I don't understand that. Apart from a few bookmarked ones
| most of the URLs I visit are embedded in one-or-another webpage (mostly
| search-engine result pages).
| | Or, said otherwise, what does "navigated to directly" mean ?
|

You go to acme.com. Acme contains code to make you load
a tracking image from statcounter.com. They also call a script
from googletagmanager.com. Maybe they have an iframe that
loads images from acmecdn.com. You visited acme.com
intentionally, but you actually visited 4 domains in all.

A few years ago, this didn't really happen. Iframes were considered
risky and in bad taste. Calling script from other domains would
have been considered an intrusion and betrayal of trust. Not only
on privacy grounds but also because cross-site scripting is a common
attack method. Today people think nothing of letting a website load
script and content from a dozen other sites. You did very much visit those
other domains. In the case of iframes you've actually loaded heir website
into a browser window, even though you may never see it. That's how
Facebook managed to set 1st party cookies on non-FB websites.
They put their logo image in an iframe. You see a small image, but it's
actually a browser window.

Newyana2,

> | Or, said otherwise, what does "navigated to directly" mean ?
>
> You go to acme.com. Acme contains code to make you load
> a tracking image from statcounter.com.

Ah, so the clicked-on ones are considered to be "navigated to directly".
Gotya.

And as far as I can tell the typed/pasted ones (into the URL bar) will be
there as well.

> You visited acme.com intentionally, but you actually visited 4
> domains in all.

I've got the "tell me and let me choose to proceede or not" setting enabled
for redirects. As a result I'm made aware of such shennigans, and most
always abort them.

> A few years ago, this didn't really happen. Iframes were
> considered risky and in bad taste.

As far as I'm concerned, they stil are.

> In the case of iframes you've actually loaded heir website
> into a browser window, even though you may never see it. That's
> how Facebook managed to set 1st party cookies on non-FB websites.
> They put their logo image in an iframe. You see a small image, but
> it's actually a browser window.

As a result of an (by default) "first-party resources only" plugin I've got
installed none of those iframes or tracker pixels get loaded. Nor most ads
for that matter. :-)

Regards,
Rudy Wieser

"R.Wieser" <address@is.invalid> wrote

| > You visited acme.com intentionally, but you actually visited 4
| > domains in all.
| | I've got the "tell me and let me choose to proceede or not" setting
enabled
| for redirects. As a result I'm made aware of such shennigans, and most
| always abort them.
| Redirects are different. That's when the server tells you that
what you're looking for is elsewhere. If the webpage is callling in
fonts from Google or ads from Doubleclick (Google), or a captcha
from gstatic.com (Google, again), or an image from their CDN
partner, those are all part of loading the page. With massive CDN
services like Akamai, you don't even have the browser officially
calling that site. If you run a sniffer you'll see Akamai connections,
but there's some kind of pass-through happening. Even putting
Akamai in a HOSTS file will have no effect because there's no DNS
lookup.

https://www.adexchanger.com/data-exchanges/eating-the-cookie-pixel-free-audience-targeting-now-available-says-akamai-cto-afergan/

The original article about this in 2010 (WSJ blocked content)
explained that Akamai is accessing 15-30% of Internet traffic and
decided to make extra
money by simply monitoring that traffic to follow people around,
then sell that data to advertisers. And that was 15 years ago.
At this point, web beacons may be all but replaced by such giant
spyware snoops, rendering the Internet into a shopping mall with
its own cameras.

I suppose it's something that really needs to be dealt with. What
Akamai and even Google are doing should be illegal. Companies are
using every possible trick to circumvent privacy. But who cares or
understands the problem well enough? Senators Ed Markey and Ron
Wyden, along with Berners-Lee. That's about it.

The original plan was that a page was basically static (even with
javascript), it all came from one server, and cookies were designed
to be readable only by the originating domain. With script now
being executable, calling back to the server, while Google, FB and
their ilk are getting their files loaded on nearly every website, and
sleazeball operations like Akamai are helping turn the Internet into a
commercial venue, it becomes easy to track peoples' activities online
in a very accurate, comprehensive way.

Microsoft started the mess with ActiveX. Up until then, js
was just for special effects and webpages always loaded from on
server. At that time a webpage of 100KB total was too big to want to
load, anyway. Then things like XMLHttpRequest
allowed a webpage to be functional and interactive, calling back
to the server. Then we got "supercookie" storage and now we're
getting compilable script. It's all moving toward webpages
essentially being large software programs to run on kiosk OSs, and
tied into numerous spyware/ad/datamining operations at every turn.

Tim Berners-Lee has repeatedly warned about this takeover:

https://webfoundation.org/2024/03/marking-the-webs-35th-birthday-an-open-letter/

But there isn't any easy way to stop is as long as people want
convenient online services and companies are making money
providing those services. Though there is an interesting article
at Atlantic this month, by one of the authors of The Coddling
of the American Mind that described the genesis of what might
be generally classed as wokist snowflakes. He's part of a growing
set of voices that's beginning to see beyond the Jetson-esque
fetish with online services and tech generally:

https://www.theatlantic.com/technology/archive/2024/03/teen-childhood-smartphone-use-mental-health-effects/677722/

The author, Haidt, is especially focusing on children, who have
been growing up in an anti-social atmosphere of cellphone
addiction, not learning the basics of human social interaction, going
from stressed out pre-teen to medicated, therapized young adults
who increasingly identify themselves as "ADHD", "traumatized", etc.
We've create a generation of fulltime "healing" shoppers.

| > A few years ago, this didn't really happen. Iframes were
| > considered risky and in bad taste.
| | As far as I'm concerned, they stil are.
|

Yes. And so is javascript. What changed is that both are
very useful for spyware and data collection. At one point about
10% of browsers had javascript disabled and it was on its
way out. But then targetted ads happened. Now many sites
are deliberately broken unless you allow the script for spying
and dynamic ad-loading purposes.

| As a result of an (by default) "first-party resources only" plugin I've
got
| installed none of those iframes or tracker pixels get loaded. Nor most
ads
| for that matter. :-)
|

I do similar with a HOSTS file. But most webpages are increasingly
broken in that respect. When acme.com loads CSS from acmecdn.com,
and images from imgacme.com, the whole idea of 1st party gets blurred.

I find, for example, that just adjusting NoScript can get very
complicated.
I have to allow 4 different Netflix domains to see videos, while blocking
the sleaze. It's not always easy to figure out what's needed for a page
to work.

Just recently I signed up for Starz. At first it worked fine. Then one day
Starz was calling in script from several outside trackers. I couldn't get
the
site to work without them. So I cancelled the service. For most people, such
a profound change to their business strategy is not seen. Starz
apparently decided to start a spyware business on the side and that's all
but invisible.

So the approach that you and I use is good, but it's an increasingly
limited strategy.