Skip to content

Converted GazeInteraction project from C++ to C#. #3427

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
21 commits merged into from
Oct 6, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
910de59
Converted GazeInteraction project for C++ to C#.
azchohfi Aug 13, 2020
2a1ab20
Merge branch 'master' into gazeCS
azchohfi Aug 24, 2020
934c810
Merge branch 'master' into gazeCS
azchohfi Sep 1, 2020
dcdf649
Addressed PR feedback.
azchohfi Sep 2, 2020
83fe4dd
Merge branch 'master' into gazeCS
azchohfi Sep 10, 2020
f49bf83
Merge branch 'master' into gazeCS
azchohfi Sep 15, 2020
ddd70aa
Merge branch 'master' into gazeCS
azchohfi Sep 16, 2020
3a9c26d
Merge branch 'master' into gazeCS
azchohfi Sep 22, 2020
5e4f30e
Add text for Gaze sample Button to make it easier to focus on.
michael-hawker Sep 22, 2020
a9a391c
Fix logic that was preventing progress reporting on Gaze events
michael-hawker Sep 22, 2020
37639f8
Fix null exception crash when removing button from Gaze sample page
michael-hawker Sep 22, 2020
bc37162
Merge branch 'master' into gazeCS
azchohfi Sep 24, 2020
6b5ce95
Merge branch 'master' into gazeCS
azchohfi Sep 25, 2020
4fa869b
Bump gaze interaction target version.
azchohfi Sep 25, 2020
bfdc1f6
Comment extra debug line
michael-hawker Sep 28, 2020
05950af
Fixed initialization order of GazeInput's static properties.
azchohfi Sep 28, 2020
a6bf4b7
Merge branch 'gazeCS' of https://github.com/windows-toolkit/WindowsCo…
azchohfi Sep 28, 2020
19722d8
Merge branch 'master' into gazeCS
azchohfi Sep 29, 2020
6119b93
Changing back GazeSettingsHelper.RetrieveSharedSettings to return IAs…
azchohfi Sep 29, 2020
91c8704
Merge branch 'master' into gazeCS
michael-hawker Oct 6, 2020
1a28ca7
Merge branch 'master' into gazeCS
michael-hawker Oct 6, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions GazeInputTest/GazeInputTest.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,8 @@
</PackageReference>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Microsoft.Toolkit.UWP.Input.GazeInteraction\Microsoft.Toolkit.Uwp.Input.GazeInteraction.vcxproj">
<Project>{a5e98964-45b1-442d-a07a-298a3221d81e}</Project>
<ProjectReference Include="..\Microsoft.Toolkit.Uwp.Input.GazeInteraction\Microsoft.Toolkit.Uwp.Input.GazeInteraction.csproj">
<Project>{5bf75694-798a-43a0-8150-415de195359c}</Project>
<Name>Microsoft.Toolkit.Uwp.Input.GazeInteraction</Name>
</ProjectReference>
</ItemGroup>
Expand Down
10 changes: 5 additions & 5 deletions GazeInputTest/Properties/AssemblyInfo.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
using System.Reflection;
using System.Runtime.InteropServices;

// General Information about an assembly is controlled through the following
// General Information about an assembly is controlled through the following
// set of attributes. Change these attribute values to modify the information
// associated with an assembly.
[assembly: AssemblyTitle("GazeInputTest")]
Expand All @@ -20,13 +20,13 @@
// Version information for an assembly consists of the following four values:
//
// Major Version
// Minor Version
// Minor Version
// Build Number
// Revision
//
// You can specify all the values or you can default the Build and Revision Numbers
// You can specify all the values or you can default the Build and Revision Numbers
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
//[assembly: AssemblyVersion("1.0.0.0")]
//[assembly: AssemblyFileVersion("1.0.0.0")]
// [assembly: AssemblyVersion("1.0.0.0")]
// [assembly: AssemblyFileVersion("1.0.0.0")]
[assembly: ComVisible(false)]
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Windows.UI.Xaml;
using Windows.UI.Xaml.Automation.Peers;
using Windows.UI.Xaml.Controls;

namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction
{
internal class ComboBoxItemGazeTargetItem : GazeTargetItem
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a ComboBox to the example:

        <ComboBox SelectedIndex="0" Margin="100,100">
          <x:String>One Thing</x:String>
          <x:String>Two Things</x:String>
          <x:String>Three Things</x:String>
          <x:String>Four Things</x:String>
        </ComboBox>

However, while it opens the combobox, it doesn't show the cursor or allow me to invoke item selection with it. Is that because we're missing the XamlRoot now or something else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is related to XamlRoot. Not sure what is happening. What was the behavior on the old version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old version wasn't working for me at all, so I couldn't compare... @peteams was the cursor supposed to move to the pop-up element and show in the drop-down?

Copy link
Member

@michael-hawker michael-hawker Sep 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, so investigated this a bit. The old code did a better job with the highlight rectangles on the ComboBox, even though I've fixed that now in the C#, the code is the same within the GazeTargetItem.RaiseProgressEvent method, so I'm not sure why it behaves differently, though maybe has to do with how the visual tree is searched in that logic? Doesn't seem to make a difference using x:String elements vs ComboBoxItem elements directly in my example above.

I am basically noticing that sometimes it does find the ComboBoxItem and can select it, other times it shows the highlight rectangle briefly, but most times there's no visual indications (the cursor is behind the pop-up too), and it seems like the UIElement selector is grabbing the 'TextBlock' element within the ComboBoxItem instead of looking up and grabbing the ComboBoxItem itself, this I think is the main thing breaking the logic here. Will try and poke a bit more, but may be easier to have someone from @peteams group take a look and help us with this fix.

Since this also wasn't working great in the C++ version, I think we should just add it to our #3497 list or a separate issue, and not block this PR. I think the PR should be good to go otherwise.

{
internal ComboBoxItemGazeTargetItem(UIElement element)
: base(element)
{
}

internal override void Invoke()
{
var peer = FrameworkElementAutomationPeer.FromElement(TargetElement);
var comboBoxItemAutomationPeer = peer as ComboBoxItemAutomationPeer;
var comboBoxItem = (ComboBoxItem)comboBoxItemAutomationPeer.Owner;

AutomationPeer ancestor = comboBoxItemAutomationPeer;
var comboBoxAutomationPeer = ancestor as ComboBoxAutomationPeer;
while (comboBoxAutomationPeer == null)
{
ancestor = ancestor.Navigate(AutomationNavigationDirection.Parent) as AutomationPeer;
comboBoxAutomationPeer = ancestor as ComboBoxAutomationPeer;
}

comboBoxItem.IsSelected = true;
comboBoxAutomationPeer.Collapse();
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using System.ComponentModel;

namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction
{
/// <summary>
/// This parameter is passed to the GazeElement::Invoked event and allows
/// the application to prevent default invocation when the user dwells on a control
/// </summary>
public sealed class DwellInvokedRoutedEventArgs : HandledEventArgs
{
internal DwellInvokedRoutedEventArgs()
{
}
}
}

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using System;
using System.ComponentModel;

namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction
{
/// <summary>
/// This parameter is passed to the GazeElement.DwellProgressFeedback event. The event is fired to inform the application of the user's progress towards completing dwelling on a control
/// </summary>
public sealed class DwellProgressEventArgs : HandledEventArgs
{
/// <summary>
/// Gets an enum that reflects the current state of dwell progress
/// </summary>
public DwellProgressState State { get; }

/// <summary>
/// Gets a value between 0 and 1 that reflects the fraction of progress towards completing dwell
/// </summary>
public double Progress { get; }

internal DwellProgressEventArgs(DwellProgressState state, TimeSpan elapsedDuration, TimeSpan triggerDuration)
{
State = state;
Progress = ((double)elapsedDuration.Ticks) / triggerDuration.Ticks;
}
}
}

This file was deleted.

32 changes: 32 additions & 0 deletions Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellProgressState.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction
{
/// <summary>
/// An enum that reflects the current state of progress towards dwell when a user is focused on a control
/// </summary>
public enum DwellProgressState
{
/// <summary>
/// User is not looking at the control
/// </summary>
Idle,

/// <summary>
/// Gaze has entered control but we're not yet showing progress.
/// </summary>
Fixating,

/// <summary>
/// User is continuing to focus on a control with an intent to dwell and invoke
/// </summary>
Progressing,

/// <summary>
/// User has completed dwelling on a control
/// </summary>
Complete
}
}
34 changes: 0 additions & 34 deletions Microsoft.Toolkit.Uwp.Input.GazeInteraction/DwellProgressState.h

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Windows.UI.Xaml;
using Windows.UI.Xaml.Automation;
using Windows.UI.Xaml.Automation.Peers;
using Windows.UI.Xaml.Automation.Provider;

namespace Microsoft.Toolkit.Uwp.Input.GazeInteraction
{
internal class ExpandCollapsePatternGazeTargetItem : GazeTargetItem
{
internal ExpandCollapsePatternGazeTargetItem(UIElement element)
: base(element)
{
}

internal override void Invoke()
{
var peer = FrameworkElementAutomationPeer.FromElement(TargetElement);
var provider = peer.GetPattern(PatternInterface.ExpandCollapse) as IExpandCollapseProvider;
switch (provider.ExpandCollapseState)
{
case ExpandCollapseState.Collapsed:
provider.Expand();
break;

case ExpandCollapseState.Expanded:
provider.Collapse();
break;
}
}
}
}
80 changes: 0 additions & 80 deletions Microsoft.Toolkit.Uwp.Input.GazeInteraction/GazeCursor.cpp

This file was deleted.

Loading