-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Converted GazeInteraction project from C++ to C#. #3427
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
910de59
Converted GazeInteraction project for C++ to C#.
azchohfi 2a1ab20
Merge branch 'master' into gazeCS
azchohfi 934c810
Merge branch 'master' into gazeCS
azchohfi dcdf649
Addressed PR feedback.
azchohfi 83fe4dd
Merge branch 'master' into gazeCS
azchohfi f49bf83
Merge branch 'master' into gazeCS
azchohfi ddd70aa
Merge branch 'master' into gazeCS
azchohfi 3a9c26d
Merge branch 'master' into gazeCS
azchohfi 5e4f30e
Add text for Gaze sample Button to make it easier to focus on.
michael-hawker a9a391c
Fix logic that was preventing progress reporting on Gaze events
michael-hawker 37639f8
Fix null exception crash when removing button from Gaze sample page
michael-hawker bc37162
Merge branch 'master' into gazeCS
azchohfi 6b5ce95
Merge branch 'master' into gazeCS
azchohfi 4fa869b
Bump gaze interaction target version.
azchohfi bfdc1f6
Comment extra debug line
michael-hawker 05950af
Fixed initialization order of GazeInput's static properties.
azchohfi a6bf4b7
Merge branch 'gazeCS' of https://github.com/windows-toolkit/WindowsCo…
azchohfi 19722d8
Merge branch 'master' into gazeCS
azchohfi 6119b93
Changing back GazeSettingsHelper.RetrieveSharedSettings to return IAs…
azchohfi 91c8704
Merge branch 'master' into gazeCS
michael-hawker 1a28ca7
Merge branch 'master' into gazeCS
michael-hawker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
36 changes: 36 additions & 0 deletions
36
Microsoft.Toolkit.Uwp.Input.GazeInteraction/ComboBoxItemGazeTargetItem.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
using Windows.UI.Xaml; | ||
using Windows.UI.Xaml.Automation.Peers; | ||
using Windows.UI.Xaml.Controls; | ||
|
||
namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction | ||
{ | ||
internal class ComboBoxItemGazeTargetItem : GazeTargetItem | ||
{ | ||
internal ComboBoxItemGazeTargetItem(UIElement element) | ||
: base(element) | ||
{ | ||
} | ||
|
||
internal override void Invoke() | ||
{ | ||
var peer = FrameworkElementAutomationPeer.FromElement(TargetElement); | ||
var comboBoxItemAutomationPeer = peer as ComboBoxItemAutomationPeer; | ||
var comboBoxItem = (ComboBoxItem)comboBoxItemAutomationPeer.Owner; | ||
|
||
AutomationPeer ancestor = comboBoxItemAutomationPeer; | ||
var comboBoxAutomationPeer = ancestor as ComboBoxAutomationPeer; | ||
while (comboBoxAutomationPeer == null) | ||
{ | ||
ancestor = ancestor.Navigate(AutomationNavigationDirection.Parent) as AutomationPeer; | ||
comboBoxAutomationPeer = ancestor as ComboBoxAutomationPeer; | ||
} | ||
|
||
comboBoxItem.IsSelected = true; | ||
comboBoxAutomationPeer.Collapse(); | ||
} | ||
} | ||
} |
19 changes: 19 additions & 0 deletions
19
Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellInvokedRoutedEventArgs.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
using System.ComponentModel; | ||
|
||
namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction | ||
{ | ||
/// <summary> | ||
/// This parameter is passed to the GazeElement::Invoked event and allows | ||
/// the application to prevent default invocation when the user dwells on a control | ||
/// </summary> | ||
public sealed class DwellInvokedRoutedEventArgs : HandledEventArgs | ||
{ | ||
internal DwellInvokedRoutedEventArgs() | ||
{ | ||
} | ||
} | ||
} |
28 changes: 0 additions & 28 deletions
28
Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellInvokedRoutedEventArgs.h
This file was deleted.
Oops, something went wrong.
31 changes: 31 additions & 0 deletions
31
Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellProgressEventArgs.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
using System; | ||
using System.ComponentModel; | ||
|
||
namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction | ||
{ | ||
/// <summary> | ||
/// This parameter is passed to the GazeElement.DwellProgressFeedback event. The event is fired to inform the application of the user's progress towards completing dwelling on a control | ||
/// </summary> | ||
public sealed class DwellProgressEventArgs : HandledEventArgs | ||
{ | ||
/// <summary> | ||
/// Gets an enum that reflects the current state of dwell progress | ||
/// </summary> | ||
public DwellProgressState State { get; } | ||
|
||
/// <summary> | ||
/// Gets a value between 0 and 1 that reflects the fraction of progress towards completing dwell | ||
/// </summary> | ||
public double Progress { get; } | ||
|
||
internal DwellProgressEventArgs(DwellProgressState state, TimeSpan elapsedDuration, TimeSpan triggerDuration) | ||
{ | ||
State = state; | ||
Progress = ((double)elapsedDuration.Ticks) / triggerDuration.Ticks; | ||
} | ||
} | ||
} |
43 changes: 0 additions & 43 deletions
43
Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellProgressEventArgs.h
This file was deleted.
Oops, something went wrong.
32 changes: 32 additions & 0 deletions
32
Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellProgressState.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction | ||
{ | ||
/// <summary> | ||
/// An enum that reflects the current state of progress towards dwell when a user is focused on a control | ||
/// </summary> | ||
public enum DwellProgressState | ||
{ | ||
/// <summary> | ||
/// User is not looking at the control | ||
/// </summary> | ||
Idle, | ||
|
||
/// <summary> | ||
/// Gaze has entered control but we're not yet showing progress. | ||
/// </summary> | ||
Fixating, | ||
|
||
/// <summary> | ||
/// User is continuing to focus on a control with an intent to dwell and invoke | ||
/// </summary> | ||
Progressing, | ||
|
||
/// <summary> | ||
/// User has completed dwelling on a control | ||
/// </summary> | ||
Complete | ||
} | ||
} |
34 changes: 0 additions & 34 deletions
34
Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellProgressState.h
This file was deleted.
Oops, something went wrong.
35 changes: 35 additions & 0 deletions
35
Microsoft.Toolkit.Uwp.Input.GazeInteraction/ExpandCollapsePatternGazeTargetItem.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
// Licensed to the .NET Foundation under one or more agreements. | ||
// The .NET Foundation licenses this file to you under the MIT license. | ||
// See the LICENSE file in the project root for more information. | ||
|
||
using Windows.UI.Xaml; | ||
using Windows.UI.Xaml.Automation; | ||
using Windows.UI.Xaml.Automation.Peers; | ||
using Windows.UI.Xaml.Automation.Provider; | ||
|
||
namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction | ||
{ | ||
internal class ExpandCollapsePatternGazeTargetItem : GazeTargetItem | ||
azchohfi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
{ | ||
internal ExpandCollapsePatternGazeTargetItem(UIElement element) | ||
: base(element) | ||
{ | ||
} | ||
|
||
internal override void Invoke() | ||
{ | ||
var peer = FrameworkElementAutomationPeer.FromElement(TargetElement); | ||
var provider = peer.GetPattern(PatternInterface.ExpandCollapse) as IExpandCollapseProvider; | ||
switch (provider.ExpandCollapseState) | ||
{ | ||
case ExpandCollapseState.Collapsed: | ||
provider.Expand(); | ||
break; | ||
|
||
case ExpandCollapseState.Expanded: | ||
provider.Collapse(); | ||
break; | ||
} | ||
} | ||
} | ||
} |
80 changes: 0 additions & 80 deletions
80
Microsoft.Toolkit.Uwp.Input.GazeInteraction/GazeCursor.cpp
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a ComboBox to the example:
However, while it opens the combobox, it doesn't show the cursor or allow me to invoke item selection with it. Is that because we're missing the
XamlRoot
now or something else?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is related to XamlRoot. Not sure what is happening. What was the behavior on the old version?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The old version wasn't working for me at all, so I couldn't compare... @peteams was the cursor supposed to move to the pop-up element and show in the drop-down?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, so investigated this a bit. The old code did a better job with the highlight rectangles on the ComboBox, even though I've fixed that now in the C#, the code is the same within the
GazeTargetItem.RaiseProgressEvent
method, so I'm not sure why it behaves differently, though maybe has to do with how the visual tree is searched in that logic? Doesn't seem to make a difference usingx:String
elements vsComboBoxItem
elements directly in my example above.I am basically noticing that sometimes it does find the ComboBoxItem and can select it, other times it shows the highlight rectangle briefly, but most times there's no visual indications (the cursor is behind the pop-up too), and it seems like the UIElement selector is grabbing the 'TextBlock' element within the ComboBoxItem instead of looking up and grabbing the ComboBoxItem itself, this I think is the main thing breaking the logic here. Will try and poke a bit more, but may be easier to have someone from @peteams group take a look and help us with this fix.
Since this also wasn't working great in the C++ version, I think we should just add it to our #3497 list or a separate issue, and not block this PR. I think the PR should be good to go otherwise.