Changing the session history spec to converge on historical & modern browser behaviour

[jakearchibald]

I'd like to update session history traversal to cater for multiple browsing contexts, and specify features that are somewhat shared by all browsers but missing from the spec. Unfortunately no browser behaves exactly the same, so I've tried to pick the common parts, then a couple of outlying behaviours that seem 'right'. I'd like to get some rough agreement on this before proceeding.

• Moving between browsing contexts at a top level (eg COOP) shouldn't lose any session history.
• Going back to a session history item should reuse the browsing context it used previously, if that browsing context is still in active use.
• Session history items should be discarded when nested contexts are discarded. Firefox currently does this, and it most closely matches the current spec.
• Moving a frame should discard its session history, since moving involves removing, and removing involves discarding the browsing context. Again this is Firefox/spec behaviour.
• Adding an item to session history should clear all current items of session history after the current point. This should be done in the joint session history. All browsers do this, but it isn't in the spec.
• When a document is discarded, the session history of its nested contexts need to be stored somehow, so it can be restored when the user navigates back. All browsers do some version of this.

When storing/reconnecting nested session history:

• A frame's name should be used when reconnecting nested session history from one document to another (reloaded) document. Chrome/Safari currently do this.
• If a frame doesn't have a name, it's connection order should be used to associate frames from one document to another (reloaded) document. All browsers do this, but I'm sure there's some inconsistencies around timing.
• Frames created with JS, should be included in this process, except frames created after DOMContentLoaded. Safari sort-of does this, but it will go beyond DOMContentLoaded, but things get pretty unreliable after that point. It seems to make sense to have a clear 'deadline' for session history restoration, after which it can be discarded.
• If an item of session history cannot be reconnected, it should be discarded. Firefox currently does this.

My hope is this would resolve (at least large parts of) the following:

• Impact of browsing context group switch during navigation #5350
• User agents, session history, browsing contexts, and agent clusters #4782
• Consider rewriting session history to follow closer what implementations do using list of trees #1454
• Scroll restoration only talks about child browsing contexts #5103

Implementers and spec folks: Are you happy with the above behaviour becoming 'the way'?

Current browser behaviour

I poked the nightlies/technical previews of Chrome, Safari, and Firefox using this test page to try and figure out how session history worked. Every browser behaves differently, so in order to specify a behaviour we need to pick the best from each browser + the current spec and compromise.

Session history and inner browsing contexts

All browsers seem to agree that session history for a 'tab' includes navigations applied to frames. The back/forward buttons treat this as a flat list that can be traversed. The spec kinda defines this with the joint session history, which is a 'getter' on the top level browsing context, which combines the history of itself and all child browsing contexts, in chronological order.

However, the way this actually works differs between Chrome, Safari, and Firefox, especially after some recent Chrome changes.

Chrome

If a history item targets a browsing context that no-longer exists (eg an iframe removed from the DOM), Chrome will retain the items in session history, but they'll be a no-op when activated.

This is also the case for iframes that are 'moved' around the DOM, since moving is remove + add, and remove discards the iframe's browsing context – Chrome does not maintain a link between the old & new browsing contexts.

Safari

I haven't fully understood Safari's behaviour yet. It seems like every item in session history has some kind of “expected browsing contexts” map, and if the reality doesn't match that map, it reloads the page from the memory cache (unless the page was served with no-store, in which case it performs a full reload) and attempts to reapply iframe navigation state in the new page.

For example:

1. Page has iframe 1, which has src 1.html.
2. Navigate iframe 1 to 2.html.
3. Remove iframe 1.
4. Press back.

Safari will reload the page, and set iframe 1 to 1.html. However:

1. Page has iframe 1, which has src 1.html.
2. Navigate iframe 1 to 2.html.
3. Add another iframe to the page.
4. Press back.

Again, Safari will reload the page, which is what makes me think its logic is broader than browsing contexts involved in a given entry of session history. However:

1. Page has iframe, with name=”iframe-1”, which has src 1.html.
2. Navigate iframe to 2.html.
3. Navigate iframe to 3.html.
4. Remove iframe.
5. Create a new iframe, with name=”iframe-1”, which has src 1.html, and add it to the DOM (doesn't matter where).
6. Press back.

In this case Safari will navigate the new iframe 'back' to 2.html without reloading the page. This only works if the iframe is named. This also means session history for a named iframe appears to be preserved when it's 'moved' around the DOM, even though moving an iframe causes the inner browsing context to be discarded and a new one created.

Firefox

When an inner browsing context is discarded (removed, or moved), its session history items are also discarded. history.length is immediately updated, but some other parts of browser UI (such as the drop down on the back button) lag behind - clicking on these entries does nothing, you remain on the same session history item.

Firefox's behaviour seems best to me, and closely matches the current spec.

Navigating a context that already has 'future' history items

All browsers seem to discard all future history items in the joint session history, whereas the spec (2.otherwise.1) only removes from the current context's browsing history.

Current browser behaviour seems best here.

Changing top level browsing context on navigation

More recently, browsers will sometimes change top level browsing context as part of a navigation, which breaks session history as defined by the spec, since a navigation that changes top-level browsing context should (as defined by the spec) lose all session history, but that isn't the behaviour we want.

There's a placeholder in the spec to deal with this.

Going 'back' to pages with inner browsing contexts

All browsers seem to agree that 'deep' session history is kinda retained despite navigating the parent. For instance, if you:

1. Navigate an iframe
2. Navigate the page containing the iframe
3. Press back

…browsers will attempt to put the iframe back into its final state in terms of session history, and pressing back again will navigate it to its previous session state.

This is especially easy in browsers that can restore the previous page from the bfcache, as all the parts are still 'live'. However, the bfcache is an optimisation, and may not be used due to particular page conditions, or may be dropped due to resource constraints.

Without bfcache, browsers will still try to restore the final navigation state by ignoring the src specified on the frame, and instead using their final url from session history.

This behaviour appears to be missing from the spec. Since the joint session history is a getter, the items which reference child browsing contexts would be lost when the parent Document is discarded. Retaining session history seems important here, so the spec needs to change. However, browser behaviour isn't consistent.

The following results are based on no-bfcache, which was simulated by serving pages with Cache-Control: no-store, and adding an unload event listener to the window.

Chrome

When you navigate back to a page, Chrome will 'reconnect' iframes with session histories based on their name attribute, falling back to their connection order.

This only applies to iframes created by the parser. Any iframes created with JS (even before DOM-ready) are not reconnected with session history.

If session history items cannot be associated with particular frames (either the named iframe is missing, the parser created fewer this time, or the iframe was created with JS), the number of items in session history is unchanged, but those session items become no-ops.

Safari

Similar to Chrome, Safari will 'reconnect' iframes with session histories based on their name attribute, falling back to their connection order.

However, this includes iframes created with JS. Even if the iframe is created after DOM-ready. For instance:

1. Add iframe-1 to DOM with src 1.html
2. Navigate iframe-1 to 2.html
3. Navigate iframe-1 to 3.html
4. Navigate top level page.
5. Back. Iframe-1 isn't there, since the page is reloaded.
6. Add iframe-1 to DOM with src 1.html.
7. Back. Iframe-1 is navigated to 2.html.
8. Back. Iframe-1 is navigated to 1.html.

Although Safari can't find iframe-1 to give it its final navigation state when the top level page is reloaded, it can 'reconnect' with it when going back through the previous session states.

Safari's “reload if browsing contexts don't look as expected” behaviour, as described earlier, also applies here. In the case of JS-created iframes, this can lead to every back-button press reloading the page but not being able to find the iframe it wants to navigate.

Firefox

Firefox will 'reconnect' iframes with session histories based on their connection order. It doesn't take the iframe's name into consideration.

Like Chrome, this only applies to iframes created by the parser. Any iframes created with JS (even before DOM-ready) are not reconnected with session history.

If session history items cannot be associated with particular frames (either the named iframe is missing, the parser created fewer this time, or the iframe was created with JS), those session history items are discarded, and history.length is immediately updated.

Specifying the history as a 'timeline'

In the current spec, the history timeline is derived from independent session history 'current entries', where the single history timeline has to be enforced through algorithms. I'd like to flip this around so the browsing session is in charge of the current history position.

I've made a little 10 minute presentation where I go through the spec changes at a high level https://www.youtube.com/watch?v=nZb0U3rFQXw.

You can think of session history as a timeline:

Step 0	Step 1	Step 2	Step 3	Step 4
1.html	2.html			3.html
	iframe1.html		bar.html
	iframe2.html	foo.html

(ignore the cell background colours, they're GitHub's default styles)

This example has 5 steps of session history:

1. Top level navigation to 1.html
2. Top level navigation to 2.html, which contains two iframes, one pointing to iframe1.html, the other to iframe2.html.
3. Navigate the second iframe to foo.html.
4. Navigate the first iframe to bar.html.
5. Top level navigation to 3.html.

In this model, each session history item is given a step number. Some history items will share a step number – in the example above there are three history items with step number 1.

The browsing session will have a 'current step', representing the current point in the timeline. The intended 'current' session history item for all navigables in a browsing session can be derived from their session history and the browsing session's current step.

For example, if the current step is 2:

Step 0	Step 1	Step 2	Step 3	Step 4
1.html	2.html			3.html
	iframe1.html		bar.html
	iframe2.html	foo.html

It's easy to derive the current history item of the top level and all the child navigables.

Then, if the second iframe is removed:

Step 0	Step 1	Step 2	Step 3	Step 4
1.html	2.html			3.html
	iframe1.html		bar.html

The iframe and its session history items are simply removed. Rather than try and remove step 2 at this point, I think it's better to handle empty steps at navigation time. For example, to go 'back':

1. Let step be the browsing session's current history step.
   Which would be 2 in this case.
2. While there isn't a session history item with step step deeply within the current browsing session, decrement step.
   Which would take step down to 1.
3. Set the browsing session's current history step to step - 1.
   Which would set the current step to 0.

If, instead, the iframe was navigated to 'hello.html':

1. Let step be the browsing session's current history step.
   Which would be 2 in this case.
2. Remove all session history items, deeply, with a step greater than step.
   This removes the bar.html and 3.html history entries.
3. Add a new history item to the iframe's navigable's session history with step step + 1.
4. Set the browsing session's current history step to step + 1.

Resulting in:

Step 0	Step 1	Step 2	Step 3
1.html	2.html
	iframe1.html		hello.html

If a new navigable is added to the page, its initial session history step will be the same as the step of the parent navigable's current session history item. Which would be 1 in this case:

Step 0	Step 1	Step 2	Step 3
1.html	2.html
	iframe1.html		hello.html
	new-iframe.html

I think this would also solve a bug in the current entry of the joint session history, which would treat the session item in the new iframe as the current entry, whereas in this model the current history item would be the parent-most session history item with the current step (decrementing as necessary), which would be hello.html.

To get the 'length' of the overall session history, take all the session history items and return the number of unique steps.

Implementers and spec folks: Would you be happy if session history was spec'd in this way?

Spec changes

I haven't thought enough about this, so there will almost certainly be changes as I work through stuff, but here's some of my notes:

A browsing session is a top-level navigable that also has:

• A current history step, a number, initially -1.
  There's never a history item with step -1. The step will be incremented when the first history item is created.

A navigable represents something that can be navigated. Things which currently contain browsing contexts for navigation purposes will use a navigable instead, and may be null in cases where browsing contexts can be null, such as a disconnected iframe. A navigable has:

• A session history, a list of session history entries.

A session history entry has:

• Everything currently defined, except the Document object (although "other information" makes me sad).
• A history step, a number. This places this session history item on the history 'timeline' of the browsing session.
• A target browsing context, a reference to a browsing context or null. This reference if weak if restorable state is not a Document.
  Yeah, I'm unsure about making a reference weak conditionally like this. I'll look for a better way to do this. The idea is that browsing contexts can go away when they don't have documents associated with them.
• A restorable state, either a Document, a restorable session history, or null. This is the new home for the Document object currently on the session history entry.

A restorable session history allows the browser to 'reconnect' child contexts with their session history when a document is recreated for a session history item. A restorable children's session history has:

• Named children, a map where the keys are strings and the values are lists of session history entries. Used to restore frames with a defined name.
• Unnamed children, a list of lists of session history entries. Used to restore frames without a name.

When an iframe is connected for the first time before document parsing is complete, it takes its URL from the restorable session history if possible, and falls back to its src. A weak reference to it is added to a list of navigables that may be serialised as part of restorable session history when the parent Document is discarded. TODO: I'm not yet sure of the exact timing of this, eg when is an iframe associated with its name in terms of history restoration?

In my model, an iframe's current browsing context is only accessible through its session history. @domenic doesn't think this will work, but I'd like to see how far I get with this 'pure' model before unravelling it a bit.

Before I spend more time figuring this out, am I at least heading in the right direction?

[jakearchibald]

Moving my complaints about history.length to #2018 (comment)

[muodov]

This is great, we've been struggling with these compat issues at Surfly a few years back. Is there something I could help with? Perhaps contributing a few WPT tests?

Also, I wonder if these points are worth extra research:

• document contexts created with window.open(). Should/do they create separate history stacks?
• should there be any differences between hops triggered by History API methods and browser back/forward buttons? I suppose there might be some security concerns?

[rakina]

This would address most of the things we wanted on #5350 (comment)! One thing not covered yet: restoring states/relationships (names, scripting relationships, whether it's an auxiliary browsing context/not, WindowProxy) when we've already deleted a browsing context and we're navigating back to it.

I think the idea proposed at the time was to save the states/relationship within the history entry. Do you think that will work? Or we can make browsing contexts live forever, if there's nothing preventing us from doing that? (In Chrome, I think that's how it's implemented - we save SiteInstance in FrameNavigationEntry so the states/relationships etc are always restorable)

[jakearchibald]

@muodov

This is great, we've been struggling with these compat issues at Surfly a few years back. Is there something I could help with? Perhaps contributing a few WPT tests?

Ohh, that would be great. It might be too soon right now, but I'll let you know.

Also, I wonder if these points are worth extra research:

• document contexts created with window.open(). Should/do they create separate history stacks?

As far as I know they create separate history stacks.

• should there be any differences between hops triggered by History API methods and browser back/forward buttons? I suppose there might be some security concerns?

I think there are some cases where the history API is a no-op if it would impact something outside of the context it was called from, eg if it's called from a sandboxed iframe.

[jakearchibald]

@rakina

This would address most of the things we wanted on #5350 (comment)! One thing not covered yet: restoring states/relationships (names

I've started playing around with ideas for that. Instead of "target browsing context" I think it'll be "restorable browsing context", where it can be a browsing context, or something that can be used to recreate the browsing context, and ensure that the recreated browsing context is used across all the history entries that previously shared a context.

In my sketches this was just a unique ID, but I can make it something that includes other state like names.

scripting relationships

I haven't read up on this yet, but I'll see if I can include it.

whether it's an auxiliary browsing context/not

I haven't thought too much about this. Is my mental model here correct:

• We may want to restore an auxiliary/top-level browsing context relationship when the browser opens.
• If a window is opened as auxiliary, but then navigated to a page isolated with COOP/COEP, we now have a browsing session that contains history entries that include both auxiliary and non-auxiliary browsing contexts.

WindowProxy) when we've already deleted a browsing context and we're navigating back to it.

Is behaviour in this case agreed? As in, should the window proxy become reconnected to the new browsing context?

I think the idea proposed at the time was to save the states/relationship within the history entry. Do you think that will work? Or we can make browsing contexts live forever, if there's nothing preventing us from doing that? (In Chrome, I think that's how it's implemented - we save SiteInstance in FrameNavigationEntry so the states/relationships etc are always restorable)

I assumed that making browsing contexts live forever would be problematic for memory, but maybe it's light enough once all the active parts are removed? Especially since it'll no longer own session history.

[rakina]

I've started playing around with ideas for that. Instead of "target browsing context" I think it'll be "restorable browsing context", where it can be a browsing context, or something that can be used to recreate the browsing context, and ensure that the recreated browsing context is used across all the history entries that previously shared a context.

In my sketches this was just a unique ID, but I can make it something that includes other state like names.

For names specifically, I just checked, it looks like whether we want to restore that or not is still unclear. Arthur Hemery (@ahemery?) from COOP team (cc @ParisMeuleman @clamy) wrote a doc on the current handling of window.names in chrome, and it looks like we actually don't store it on history entries. There's also a related bug (quite old).

Edit: From #5679, though, it looks like Firefox and Safari are already restoring browsing context names?

whether it's an auxiliary browsing context/not

I haven't thought too much about this. Is my mental model here correct:

• We may want to restore an auxiliary/top-level browsing context relationship when the browser opens.
• If a window is opened as auxiliary, but then navigated to a page isolated with COOP/COEP, we now have a browsing session that contains history entries that include both auxiliary and non-auxiliary browsing contexts.

Yes, I think if we the next page is isolated with COOP we would change to a new browsing context (and browsing context group), and it won't be an auxilliary BC (as it isn't related to another top level BC anymore). When restoring the old page, its browsing context should still be an auxilliary BC (maybe should be tied to whether or not we restore WindowProxy below, though)

Some parts where the bit is important: script-closable and not clearing name.

WindowProxy) when we've already deleted a browsing context and we're navigating back to it.

Is behaviour in this case agreed? As in, should the window proxy become reconnected to the new browsing context?

Hmm right. In Chrome (from local testing):

Case #1 (BC is gone, BCG stays because there's another BC alive)

1. Navigate to A1, which window.opens A2 in Window 2
2. Navigate A2 to B (Switches to new BC due to SiteIsolation - A1's WindowProxy to Window 2 should not work)
3. Navigate B back to A2. A1's WindowProxy to A2 should work again. A2'w WindowProxy to A1 (window.opener should work again).

Case #2 (BCG & BCs are gone):

1. Navigate to A1, which window.opens A2 in Window 2
2. Navigate A2 to B
3. Navigate A1 to C.
4. Navigate B back to A2 and C back to A1. A2's window.opener won't work (also A1 doesn't have any WindowProxy to A2 since it's newly loaded).

I haven't looked at the exact restoration logic in Chrome, maybe will do so soon and report back, or @csreis can comment :) Also cc @annevk @mystor who commented that maybe this is not how that would work in Firefox #5350 (comment)

I assumed that making browsing contexts live forever would be problematic for memory, but maybe it's light enough once all the active parts are removed? Especially since it'll no longer own session history.

Yeah I guess we kinda only want to make it possible to do "if we restore entry for B, and B's window.opener was A and it's still around we should restore the WindowProxys between them".

• Restoring B->A WindowProxy: Saving the window.opener in the entry would be enough.
• Restoring A->B WindowProxy: WindowProxy is implicitly defined as 1:1 with BC, so we need to do something here. When we restore B, we know about A (through saved window.opener). Maybe we should have a way to say to A that "hey, I used to be the BC you had a WindowProxy to". Maybe when we navigated away from B the first time, have A save the history entry id for B or something to actually make it possible to identify it.

[jakearchibald]

I'm trying to figure out the window proxy stuff, and it seems a little different in browsers:

Test 1

1. Navigate Tab1 to https://example.com.
2. Run const win = open(location.href); in Tab1 (with popups allowed), which opens Tab2
3. Run location.href = 'https://jakearchibald.com/' in Tab2
4. Run onmessage = console.log in Tab 2
5. Run win.postMessage('hello', '*') in Tab1
6. Run win.document in Tab1

All browsers (Chrome, Safari, Firefox) deliver the post-message (step 5) and error when trying to access the document (step 6). This seems as expected.

Test 2

1. Navigate Tab1 to https://example.com.
2. Run const win = open(location.href); in Tab1 (with popups allowed), which opens Tab2
3. Navigate Tab2 to https://jakearchibald.com using the location bar
4. Run onmessage = console.log in Tab 2
5. Run win.postMessage('hello', '*') in Tab1
6. Run win.document in Tab1
7. Run win.close() in Tab1

Chrome: Fails to deliver message (step 5). Cross-origin error in step 6. Fails to close in step 7.
Firefox: Delivers message (step 5). Cross-origin error in step 6. Closes tab in step 7.
Safari: Delivers message (step 5). Cross-origin error in step 6. Closes tab in step 7.

Chrome's behaviour here seems more secure. Since the navigation is manual, it seems like the new page should be cut-off from its opener. However, it isn't entirely cut-off, since you get the cross-origin error in step 6.

Test 3

1. Navigate Tab1 to https://example.com.
2. Run const win = open(location.href); in Tab1 (with popups allowed), which opens Tab2
3. Run location.href = 'https://first-party-test.glitch.me/coep?coep=require-corp&coop=same-origin' in Tab2 - this is a cross-origin isolated page.
4. Run onmessage = console.log in Tab 2
5. Run win.postMessage('hello', '*') in Tab1
6. Run win.document in Tab1
7. Run win.close() in Tab1
8. Press 'back' on Tab2
9. Run win.close() in Tab1

Chrome: Fails to deliver message in step 5. win.document is undefined in step 6 . Fails to close in step 7. Fails to close in step 9.
Firefox: Fails to deliver message in step 5. win.document is the initial example.com document from Tab2 (step 6). Fails to close in step 7. Fails to close in step 9.
Safari: Does not support isolation.

The initial win.document difference in Chrome and Firefox could be explained by bfcache, where both Chrome and Firefox are treating win as a proxy to the previous Tab2 page, but it doesn't have a document in Chrome because it's discarded. However, when Firefox goes back it doesn't use a bfcache entry, it reloads the page, and win remains pointing to the old page.

It makes sense to have win continue to refer to the old page if we want the window proxy to be owned by the browsing context. However, it should have reconnected when the browsers went back.

Proposal

When the user navigates the tab to a new history entry using a method outside of the document (location bar, bookmarks etc), the new page should not have ties to the opener. This could be achieved by creating a new browsing context for the history entry, which means the behaviour of tests 2 & 3 should be the same.

The window proxy would continue to be owned by the browsing context, although maybe we should prevent access to things like the document while the target page is in bfcache?

I plan to store the browsing context in the session history entry, so when the browser goes back it can reload the page in the right context.

WDYT @rakina? @annevk?

[domenic]

A quick note between meetings...

Test 2

Chrome's behaviour here seems more secure. Since the navigation is manual, it seems like the new page should be cut-off from its opener. However, it isn't entirely cut-off, since you get the cross-origin error in step 6.

I'm not sure there's a security issue here, per se: postMessage is about message passing, which should not be a security problem to allow cross-origin.

What's happening here, I'm pretty sure, is that Chrome did a BCG swap on URL bar navigation, so that puts the window into a separate browsing context group, which conceptually means you can't message it.

That is, normally when you have two different windows in different BCGs, there's no way for them to get a handle to each other, so messaging is impossible. With Chrome's BCG swap on URL bar navigation, that invariant no longer holds; we have windows with handles to each other, but in different BCGs. I guess this manifests in Chrome by having postMessage() fail, in a not-specced way.

I slightly favor the Safari and Firefox behavior here, in that I like the idea that if you have a handle to a window then you can message it. But I don't feel strongly, and if Chrome implementers have implementation concerns then that should probably outweigh my instincts.

Plus, there might be a theoretical reason why allowing cross-BCG messaging is a bad idea. I haven't processed your test 3 yet, or your proposal... Will do after the next meeting block.