Regression: DNU restore tries to load the current project from the package cache

[borgdylan]

As of the latest DNX, running dnu restore requires the nupkg for the current project to be in the package cache, something that looks pretty buggy.

dylan@ubuntu-server:/var/www/Code/dylannet/vnext/dylan.NET.K$ dnu restore
Microsoft .NET Development Utility Mono-x86-1.0.0-beta6-12207

Restoring packages for /var/www/Code/dylannet/vnext/dylan.NET.K/project.json
  CACHE http://www.myget.org/F/aspnetwebstacknightly/
  CACHE https://nuget.org/api/v2/curated-feeds/dotnetframework/
  CACHE https://www.myget.org/F/aspnetvnext/api/v2/
  CACHE  https://api.nuget.org/v3/index.json
  CACHE https://www.myget.org/F/npgsql-unstable/api/v2/
  CACHE http://www.myget.org/F/aspnetwebstacknightly/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions'
  CACHE https://nuget.org/api/v2/curated-feeds/dotnetframework/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions'
  CACHE https://www.myget.org/F/aspnetvnext/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions'
  CACHE https://nugetapiapps.blob.core.windows.net/ngx/microsoft.framework.runtime.abstractions/index.json
  CACHE https://www.myget.org/F/npgsql-unstable/api/v2/FindPackagesById()?id='Microsoft.Framework.Runtime.Abstractions'
Writing lock file /var/www/Code/dylannet/vnext/dylan.NET.K/project.lock.json
----------
System.IO.DirectoryNotFoundException: Could not find a part of the path "/home/dylan/.dnx/packages/dylan.NET.K/1.3.6/dylan.NET.K.1.3.6.nupkg.sha512".
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) <0xb5380b20 + 0x00217> in <filename unknown>:0 
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, System.String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) <0xb5380a50 + 0x0004f> in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare,int,System.IO.FileOptions,string,bool,bool,bool)
  at System.IO.StreamReader..ctor (System.String path, System.Text.Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean checkHost) <0xb51c0190 + 0x000df> in <filename unknown>:0 
  at System.IO.StreamReader..ctor (System.String path, System.Text.Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize) <0xb51c0140 + 0x00042> in <filename unknown>:0 
  at System.IO.StreamReader..ctor (System.String path, Boolean detectEncodingFromByteOrderMarks) <0xb51c0050 + 0x00046> in <filename unknown>:0 
  at System.IO.StreamReader..ctor (System.String path) <0xb51c0010 + 0x0002c> in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.IO.StreamReader:.ctor (string)
  at System.IO.File.ReadAllText (System.String path) <0xb537e6a0 + 0x0003b> in <filename unknown>:0 
  at Microsoft.Framework.PackageManager.Utils.LockFileUtils.CreateLockFileLibrary (Microsoft.Framework.Runtime.DependencyManagement.LockFileLibrary previousLibrary, IPackagePathResolver resolver, IPackage package, System.String correctedPackageName) <0xb490da58 + 0x00197> in <filename unknown>:0 
  at Microsoft.Framework.PackageManager.RestoreCommand.WriteLockFile (Microsoft.Framework.Runtime.DependencyManagement.LockFile previousLockFile, System.String projectLockFilePath, Microsoft.Framework.Runtime.Project project, System.Collections.Generic.List`1 graphItems, NuGet.PackageRepository repository, IEnumerable`1 contexts) <0xb490c7c8 + 0x00753> in <filename unknown>:0 
  at Microsoft.Framework.PackageManager.RestoreCommand+<RestoreForProject>d__70.MoveNext () <0xb700b1c8 + 0x03203> in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0xb51fbf30 + 0x00035> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0xb51faf40 + 0x000b7> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0xb51faeb0 + 0x00084> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0xb51fae60 + 0x0003f> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () <0xb54eead0 + 0x0001f> 24208 in <filename unknown>:0 
  at Microsoft.Framework.PackageManager.RestoreCommand+<>c__DisplayClass69_0+<<Execute>b__1>d.MoveNext () <0xb700aaf0 + 0x00157> in <filename unknown>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () <0xb51fbf30 + 0x00035> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) <0xb51faf40 + 0x000b7> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) <0xb51faeb0 + 0x00084> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) <0xb51fae60 + 0x0003f> in <filename unknown>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () <0xb51fae30 + 0x0001f> in <filename unknown>:0 
  at Microsoft.Framework.PackageManager.RestoreCommand+<Execute>d__69.MoveNext () <0xb701f560 + 0x0089f> in <filename unknown>:0 
----------
Restore failed
Could not find a part of the path "/home/dylan/.dnx/packages/dylan.NET.K/1.3.6/dylan.NET.K.1.3.6.nupkg.sha512".

NuGet Config files used:
    /home/dylan/.config/NuGet/NuGet.Config

Feeds used:
    http://www.myget.org/F/aspnetwebstacknightly/
    /var/www/nugetfeed
    https://nuget.org/api/v2/curated-feeds/dotnetframework/
    https://www.myget.org/F/aspnetvnext/api/v2/
    https://nugetapiapps.blob.core.windows.net/ngx/
    https://www.myget.org/F/npgsql-unstable/api/v2/

[borgdylan]

The lock file is being updated, but that exception can still concern people.

[davidfowl]

Looks like a duplicate of #2192

[davidfowl]

/cc @troydai did the build pass yet?

[borgdylan]

Since that package is a compiler provider, anything that uses it is also failing to restore and failing to build since the dependency package is not building.

[davidfowl]

Sure but that's not relevant to the bug. The goal is to help us narrow down the problem by providing us to proper repro steps. I can't even reproduce the bug right now locally. Where is that package coming from?

[borgdylan]

Its local source code.

[borgdylan]

By some magic, deleting all of the package cache and restoring, the bug is gone.

[davidfowl]

@anurse @ChengTian Based on @troydai's changes I'm guessing that you can end up with a situation like this:

Old DNX before @troydai 's fix:

.dnx\packages\MyPackage\1.0.0\MyPackage.1.0.0.0.sha512

New DNX after @troydai's fix:

.dnx\packages\MyPackage\1.0.0\MyPackage.1.0.0.sha512

Notice the normalized version folder and the sha512 file. Since we changed the format, we need to make sure that we only use the folder if it is in the format we expect. This means that we need some extra logic to search for {id} + {normalized}. {extension} at dnu restore time.

/cc @muratg This might be a large regression that we should look at ASAP.

[troydai]

Latest build on vNext is 12207, which should contain the change. The normalized version in file name is expected. I tested with FluentManager the installed package.

The issue here seems to because the old DNX installed the package using pre-2192-change name, while after upgrading to new DNX the old package is recognized, but normalized name is used to look for name.

[troydai]

I can repo this. Here's a quick repo step:
Create a project:

{
    "dependencies": { "dylan.NET.K": "1.3.5" }
}

1. Install DNX 1.0.0-beta6-12189, dnu restore.
   Notice the packages been added is under ~/.dnx/dylan.NET.k/1.3.5.0/
2. Install DNX 1.0.0-beta6-12207, do NOT clean the packages folder and dnu restore.

[Error] Can't find the sha file under ~/.dnx/dylan.NET.k/1.3.5/

I'm looking at dnu restore now to see where the existing packages is recognized. The normalized string should be used instead.

[borgdylan]

When publishing nupkgs are created with versions like 1.0.0.0 in yet the publish process looks for 1.0.0.

[troydai]

@borgdylan yes. Thanks for reminding. It is tracked in #2267. I'm working on a fix.

Thanks!