Analyzer crashes on deftype / reify in same ns

[zampino]

Reproducible by showing the following ns with clerk. Note: it won't reproduce if the defprotocol form (or the whole ns) is evaluated in the REPL before calling clerk/show!. Analysis also won't crash if the protocol is required from another ns.

(ns scratch.meta-analyzer-issues-vii)

(defprotocol Foo
  (-foo [this]))

(reify Foo
  (-foo [this] :baz))

{:via [{:type java.lang.NullPointerException,
        :message "Cannot invoke \"java.lang.Class.getName()\" because \"p1__5034_SHARP_\" is null",
        :at [clojure.tools.analyzer.jvm$_deftype$fn__5039 invoke "jvm.clj" 319]}],
 :trace [[clojure.tools.analyzer.jvm$_deftype$fn__5039 invoke "jvm.clj" 319]
         [clojure.core$mapv$fn__8535 invoke "core.clj" 6979]
         [clojure.core.protocols$iter_reduce invokeStatic "protocols.clj" 49]
         [clojure.core.protocols$fn__8230 invokeStatic "protocols.clj" 75]
         [clojure.core.protocols$fn__8230 invoke "protocols.clj" 75]
         [clojure.core.protocols$fn__8178$G__8173__8191 invoke "protocols.clj" 13]
         [clojure.core$reduce invokeStatic "core.clj" 6886]
         [clojure.core$mapv invokeStatic "core.clj" 6970]
         [clojure.core$mapv invoke "core.clj" 6970]
         [clojure.tools.analyzer.jvm$_deftype invokeStatic "jvm.clj" 319]
         [clojure.tools.analyzer.jvm$_deftype invoke "jvm.clj" 313]
         [nextjournal.clerk.analyzer$analyze_form$_deftype__56360 invoke "analyzer.clj" 140]
         [clojure.tools.analyzer.jvm$parse_reify_STAR_ invokeStatic "jvm.clj" 334]
         [clojure.tools.analyzer.jvm$parse_reify_STAR_ invoke "jvm.clj" 324]
         [clojure.tools.analyzer.jvm$parse invokeStatic "jvm.clj" 427]
         [clojure.tools.analyzer.jvm$parse invoke "jvm.clj" 424]
         [clojure.tools.analyzer$analyze_seq invokeStatic "analyzer.clj" 271]
         [clojure.tools.analyzer$analyze_seq invoke "analyzer.clj" 262]
         [clojure.tools.analyzer$eval1933$fn__1934 invoke "analyzer.clj" 63]
         [clojure.lang.MultiFn invoke "MultiFn.java" 234]
         [clojure.tools.analyzer$analyze_seq invokeStatic "analyzer.clj" 272]
         [clojure.tools.analyzer$analyze_seq invoke "analyzer.clj" 262]
         [clojure.tools.analyzer$eval1933$fn__1934 invoke "analyzer.clj" 63]
         [clojure.lang.MultiFn invoke "MultiFn.java" 234]
         [clojure.tools.analyzer$analyze invokeStatic "analyzer.clj" 115]
         [clojure.tools.analyzer$analyze invoke "analyzer.clj" 88]
         [clojure.tools.analyzer.jvm$analyze$fn__5099$fn__5104 invoke "jvm.clj" 513]
         [clojure.tools.analyzer.jvm$analyze$fn__5099 invoke "jvm.clj" 511]
         [clojure.lang.AFn applyToHelper "AFn.java" 152]
         [clojure.lang.AFn applyTo "AFn.java" 144]
         [clojure.core$apply invokeStatic "core.clj" 667]
         [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1990]
         [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1990]
         [clojure.lang.RestFn invoke "RestFn.java" 425]
         [clojure.tools.analyzer.jvm$analyze invokeStatic "jvm.clj" 500]
         [clojure.tools.analyzer.jvm$analyze invoke "jvm.clj" 480]
         [nextjournal.clerk.analyzer$analyze_form$fn__56362 invoke "analyzer.clj" 141]
         [clojure.core$with_redefs_fn invokeStatic "core.clj" 7582]
         [clojure.core$with_redefs_fn invoke "core.clj" 7566]
         [nextjournal.clerk.analyzer$analyze_form invokeStatic "analyzer.clj" 138]

[zampino]

The same happens in a plain analyzer call, when the type is not yet in context:

(clojure.tools.analyzer.jvm/analyze '(reify Foo
                                         (-foo [this] :baz)))
;; throws
;; Execution error (NullPointerException) at clojure.tools.analyzer.jvm/-deftype$fn (jvm.clj:319).
;; Cannot invoke "java.lang.Class.getName()" because "p1__5034_SHARP_" is null

Therefore I don't believe is related the changes in #386.

[teodorlu]

I believe I hit this issue too.

The full namespace where I hit the error

https://github.com/teodorlu/play.teod.eu/tree/fe70f7ed3fc6f997669b5356925f220d0c6e5038/how-much-is-that-second-dice-worth/second_dice.clj

I hit it when implementing Sample for Dice. The protocol already existed, and I needed to implement it for a new record.

REPL is run from the same folder as second_dice.clj, not the parent folder (which also contains a deps.edn file).

Observations

1. Clerk version: io.github.nextjournal/clerk {:mvn/version "0.15.957"}
2. I'm using a "bare" namespace (no dots in it): (ns second-dice ,,,), and :paths ["."] in deps.edn
3. *clojure-version* is {:major 1, :minor 11, :incremental 1, :qualifier nil}
4. In my namespace, I first define a protocol Sample, then create two records Uniform and Dice that implement the protocol
5. With a call to clerk/show! I hit a :clojure.error/cause "No implementation of method: :sample of protocol: #'second-dice/Sample found for class: second_dice.Dice". (see screenshot and attached CIDER error)
6. When I evaluate each form from top to bottom in a REPL from CIDER, I hit no errors.
7. When I call clerk/clear-cache!, then clerk/show! after, I hit no errors.
8. I tried narrowing down the error to figure out exactly went wrong, but wasn't able to reproduce with a simpler example.

Screenshot of what I'm seeing:

Full CIDER error

CIDER ERROR

  Show: Project-Only All 
  Hide: Clojure Java REPL Tooling Duplicates  (0 frames hidden)


Unhandled clojure.lang.ExceptionInfo

nextjournal.clerk/show! encountered an eval error with:

"/Users/teodorlu/dev/teodorlu/play.teod.eu/how-much-is-that-second-dice-worth/second_dice.clj"

#:nextjournal.clerk{:doc

{:nav-path "second_dice",

:blocks

[{:type :markdown,

:doc

{:type :doc,

:content

[{:type :heading,

:content

[{:type :text,

:text "How much is that second dice worth?"}],

:heading-level 1,

:attrs {:id "how-much-is-that-second-dice-worth?"}}],

:footnotes []}}

{:type :code,

:text

"(ns second-dice\n  {:nextjournal.clerk/toc true}\n  (:require\n   [nextjournal.clerk :as clerk]))",

:loc {:line 3, :end-line 6, :column 1, :end-column 35}}

{:type :markdown,

:doc

{:type :doc,

:content

[{:type :heading,

:content [{:type :text, :text "Rationale"}],

:heading-level 2,

:attrs {:id "rationale"}}

{:type :paragraph,

:content

[{:type :text,

:text

"what is the expected value of the best of two dice throws?"}]}

{:type :paragraph,

:content

[{:type :text,

:text "let's find out with monte-carlo!"}]}

{:type :paragraph,

:content

[{:type :text, :text "in which you'll learn:"}]}

{:type :bullet-list,

:content

[{:type :list-item,

:content

[{:type :plain,

:content

[{:type :text,

:text

"How to use Monte-Carlo simulation to optimize your Arcane Survivors strategy."}]}]}

{:type :list-item,

:content

[{:type :plain,

:content

[{:type :text,

:text

"How to implement Monte-Carlo simulation in Clojure."}]}]}]}

{:type :heading,

:content

[{:type :text, :text "To sample uniform data"}],

:heading-level 2,

:attrs {:id "to-sample-uniform-data"}}],

:footnotes []}}

{:type :code,

:text "213",

:loc {:line 21, :end-line 21, :column 1, :end-column 4}}

{:type :code,

:text "(defprotocol Sample\n  (sample [this n]))",

:loc

{:line 23, :end-line 24, :column 1, :end-column 21}}

{:type :code,

:text

"(defrecord Uniform [lower upper]\n  Sample\n  (sample [_this n]\n    (repeatedly n #(+ lower (rand (- upper lower))))))",

:loc

{:line 26, :end-line 29, :column 1, :end-column 55}}

{:type :code,

:text "(sample (Uniform. 1 300) 10)",

:loc

{:line 31, :end-line 31, :column 1, :end-column 29}}

{:type :code,

:text

"(defrecord Dice [sides]\n  Sample\n  (sample [_this n]\n    (repeatedly n #(inc (rand-int sides)))))",

:loc

{:line 33, :end-line 36, :column 1, :end-column 45}}

{:type :code,

:text "(comment (clerk/clear-cache!))",

:loc

{:line 38, :end-line 38, :column 1, :end-column 31}}

{:type :code,

:text

"(defn mean [xs] (float (/ (reduce + xs) (count xs))))",

:loc

{:line 40, :end-line 40, :column 1, :end-column 54}}

{:type :code,

:text "(def d6 (Dice. 6))",

:loc

{:line 42, :end-line 42, :column 1, :end-column 19}}

{:type :code,

:text

"(let [d6-sample (sample d6 1000)]\n  {:mean (mean d6-sample)\n   :min (apply min d6-sample)\n   :max (apply max d6-sample)})",

:loc

{:line 44, :end-line 47, :column 1, :end-column 32}}

{:type :code,

:text "(sample d6 10)",

:loc

{:line 49, :end-line 49, :column 1, :end-column 15}}

{:type :code,

:text "(def d20 (Dice. 20))",

:loc

{:line 51, :end-line 51, :column 1, :end-column 21}}

{:type :code,

:text "(sample d20 10)",

:loc

{:line 53, :end-line 53, :column 1, :end-column 16}}

{:type :markdown,

:doc

{:type :doc,

:content

[{:type :paragraph,

:content

[{:type :text, :text "this doesn't look "}

{:type :em,

:content [{:type :text, :text "directly"}]}

{:type :text, :text " wrong."}]}

{:type :paragraph,

:content

[{:type :text,

:text

"but we want to view histograms, not numbers with lots of decimal places!"}]}

{:type :paragraph,

:content

[{:type :text, :text "let's use Vega-Lite."}]}

{:type :heading,

:content

[{:type :text, :text "Histogram viewer, take 1"}],

:heading-level 2,

:attrs {:id "histogram-viewer,-take-1"}}],

:footnotes []}}

{:type :code,

:text

"(clerk/caption "Looks like a histogram?"\n               (clerk/vl\n                {:width 500\n                 :height 300\n                 :data {:values [{"a" "A", "b" 28}, {"a" "B", "b" 55}, {"a" "C", "b" 43},\n                                 {"a" "D", "b" 91}, {"a" "E", "b" 81}, {"a" "F", "b" 53},\n                                 {"a" "G", "b" 19}, {"a" "H", "b" 87}, {"a" "I", "b" 52}]},\n                 :mark :bar,\n                 :encoding {:x {:field "a", :type "nominal", "axis" {:labelAngle 0}},\n                            :y {:field "b", :type "quantitative"}}}))",

:loc

{:line 63, :end-line 72, :column 1, :end-column 70}}

{:type :markdown,

:doc

{:type :doc,

:content

[{:type :heading,

:content [{:type :text, :text "but ... buckets???"}],

:heading-level 2,

:attrs {:id "but-...-buckets???"}}],

:footnotes []}}

{:type :code,

:text

"(clerk/caption "Looks like a histogram?"\n (clerk/vl\n  {:width 500\n   :height 300\n   :data {:values [{"a" "A", "b" 28}, {"a" "B", "b" 55}, {"a" "C", "b" 43},\n                   {"a" "D", "b" 91}, {"a" "E", "b" 81}, {"a" "F", "b" 53},\n                   {"a" "G", "b" 19}, {"a" "H", "b" 87}, {"a" "I", "b" 52}]},\n   :mark :bar,\n   :encoding {:x {:field "a", :type "nominal", "axis" {:labelAngle 0}},\n              :y {:field "b", :type "quantitative"}}}))",

:loc

{:line 76, :end-line 85, :column 1, :end-column 56}}

{:type :code,

:text

"^{:nextjournal.clerk/visibility {:code :hide}}\n(clerk/html [:div {:style {:height "50vh"}}])",

:loc

{:line 87, :end-line 88, :column 1, :end-column 46}}],

:title "How much is that second dice worth?",

:toc

{:type :toc,

:children

[{:type :toc,

:content

[{:type :text,

:text "How much is that second dice worth?"}],

:heading-level 1,

:attrs {:id "how-much-is-that-second-dice-worth?"},

:path [:content 0],

:children

[{:type :toc,

:content [{:type :text, :text "Rationale"}],

:heading-level 2,

:attrs {:id "rationale"},

:path [:content 1]}

{:type :toc,

:content

[{:type :text, :text "To sample uniform data"}],

:heading-level 2,

:attrs {:id "to-sample-uniform-data"},

:path [:content 6]}

{:type :toc,

:content

[{:type :text, :text "Histogram viewer, take 1"}],

:heading-level 2,

:attrs {:id "histogram-viewer,-take-1"},

:path [:content 10]}

{:type :toc,

:content [{:type :text, :text "but ... buckets???"}],

:heading-level 2,

:attrs {:id "but-...-buckets???"},

:path [:content 11]}]}]},

:footnotes [],

:file

"/Users/teodorlu/dev/teodorlu/play.teod.eu/how-much-is-that-second-dice-worth/second_dice.clj",

:blob->result

{"5dsoxVNmQxeLrQc5BYuiSYpP3pPgPU"

#:nextjournal{:value smaller.Uniform,

:blob-id "5dsoxVNmQxeLrQc5BYuiSYpP3pPgPU"},

"5duBRHzaZJdte1P7SqWpWzez5h2D6e"

#:nextjournal{:value nil,

:blob-id "5duBRHzaZJdte1P7SqWpWzez5h2D6e"},

"8VxMLktXDdtVMc8iXimmtZ8NSKSLuY18iS4X8ByUYe38wJcyDLyPHV17oQkSvYDuZuMZzBdwAVhWgxupuvjsQC4uMd"

#:nextjournal{:value nil,

:blob-id

"8VxMLktXDdtVMc8iXimmtZ8NSKSLuY18iS4X8ByUYe38wJcyDLyPHV17oQkSvYDuZuMZzBdwAVhWgxupuvjsQC4uMd"},

"5dsqSVvyyeABSYbgw2YaeNe3zVVZNZ"

#:nextjournal{:value {:mean 3.482, :min 1, :max 6},

:blob-id "5dsqSVvyyeABSYbgw2YaeNe3zVVZNZ"},

"5dtGpKAVzd6dSw8dGHhKShj3zo76QR"

#:nextjournal{:value

#:nextjournal.clerk{:var-from-def

#'smaller/d6},

:blob-id "5dtGpKAVzd6dSw8dGHhKShj3zo76QR"},

"5dr3zv65J8wG2hqKjzscmsTfZcar9q"

#:nextjournal{:value

(63.94751404528239

109.61342435999796

211.8930144616428

247.4013624582594

132.50491614451082

251.8310728888618

68.69533339809387

124.25178328910593

57.57708915051512

131.46465307550073),

:blob-id "5dr3zv65J8wG2hqKjzscmsTfZcar9q"},

"8VxpH18GjYxNbx3R7fHcpHJuFTraYGmZAYfbMhhinozHi5hXSCRecGJTtk8WHo1AhJghN16Hb3nFNMfQdcbhFPUeHz"

#:nextjournal{:value Sample,

:blob-id

"8VxpH18GjYxNbx3R7fHcpHJuFTraYGmZAYfbMhhinozHi5hXSCRecGJTtk8WHo1AhJghN16Hb3nFNMfQdcbhFPUeHz",

:interned #{smaller/sample}},

"5dthtWrreajuETiyE2wMakNLShorEF"

#:nextjournal{:value smaller.Dice,

:blob-id

"5dthtWrreajuETiyE2wMakNLShorEF"}}}}

clerk.clj:   66  nextjournal.clerk/show!/fn

clerk.clj:   64  nextjournal.clerk/show!

clerk.clj:   21  nextjournal.clerk/show!

clerk.clj:   32  nextjournal.clerk/show!

clerk.clj:   21  nextjournal.clerk/show!

REPL:    1  second-dice/eval35852

REPL:    1  second-dice/eval35852

Compiler.java: 7194  clojure.lang.Compiler/eval

Compiler.java: 7149  clojure.lang.Compiler/eval

core.clj: 3215  clojure.core/eval

core.clj: 3211  clojure.core/eval

interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn

AFn.java:  152  clojure.lang.AFn/applyToHelper

AFn.java:  144  clojure.lang.AFn/applyTo

core.clj:  667  clojure.core/apply

core.clj: 1990  clojure.core/with-bindings*

core.clj: 1990  clojure.core/with-bindings*

RestFn.java:  425  clojure.lang.RestFn/invoke

interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn

main.clj:  437  clojure.main/repl/read-eval-print/fn

main.clj:  437  clojure.main/repl/read-eval-print

main.clj:  458  clojure.main/repl/fn

main.clj:  458  clojure.main/repl

main.clj:  368  clojure.main/repl

RestFn.java: 1523  clojure.lang.RestFn/invoke

interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate

interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate

interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn

AFn.java:   22  clojure.lang.AFn/run

session.clj:  218  nrepl.middleware.session/session-exec/main-loop/fn

session.clj:  217  nrepl.middleware.session/session-exec/main-loop

AFn.java:   22  clojure.lang.AFn/run

Thread.java: 1583  java.lang.Thread/run


Caused by clojure.lang.ExceptionInfo

Execution error (IllegalArgumentException) at second-dice/eval35990$fn$G

(second_dice.clj:23). No implementation of method: :sample of protocol:

#'second-dice/Sample found for class: second_dice.Dice
{:clojure.error/cause

"No implementation of method: :sample of protocol: #'second-dice/Sample found for class: second_dice.Dice",

:clojure.error/phase :execution,

:clojure.error/symbol second-dice/eval35990$fn$G,

:clojure.error/line 23,

:clojure.error/class java.lang.IllegalArgumentException,

:clojure.error/source "second_dice.clj",

:line 44,

:col 1,

:clojure.core/eval-file

"/Users/teodorlu/dev/teodorlu/play.teod.eu/how-much-is-that-second-dice-worth/second_dice.clj",

:form

(let

[d6-sample (sample d6 1000)]

{:mean (mean d6-sample),

:min (apply min d6-sample),

:max (apply max d6-sample)})}

eval.clj:  155  nextjournal.clerk.eval/eval+cache!

eval.clj:  122  nextjournal.clerk.eval/eval+cache!

eval.clj:  194  nextjournal.clerk.eval/read+eval-cached

eval.clj:  165  nextjournal.clerk.eval/read+eval-cached

eval.clj:  234  nextjournal.clerk.eval/eval-analyzed-doc/fn

PersistentVector.java:  343  clojure.lang.PersistentVector/reduce

core.clj: 6885  clojure.core/reduce

core.clj: 6868  clojure.core/reduce

eval.clj:  229  nextjournal.clerk.eval/eval-analyzed-doc

eval.clj:  226  nextjournal.clerk.eval/eval-analyzed-doc

eval.clj:  253  nextjournal.clerk.eval/+eval-results

eval.clj:  246  nextjournal.clerk.eval/+eval-results

clerk.clj:   64  nextjournal.clerk/show!/fn

clerk.clj:   64  nextjournal.clerk/show!

clerk.clj:   21  nextjournal.clerk/show!

clerk.clj:   32  nextjournal.clerk/show!

clerk.clj:   21  nextjournal.clerk/show!

REPL:    1  second-dice/eval35852

REPL:    1  second-dice/eval35852

Compiler.java: 7194  clojure.lang.Compiler/eval

Compiler.java: 7149  clojure.lang.Compiler/eval

core.clj: 3215  clojure.core/eval

core.clj: 3211  clojure.core/eval

interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn

AFn.java:  152  clojure.lang.AFn/applyToHelper

AFn.java:  144  clojure.lang.AFn/applyTo

core.clj:  667  clojure.core/apply

core.clj: 1990  clojure.core/with-bindings*

core.clj: 1990  clojure.core/with-bindings*

RestFn.java:  425  clojure.lang.RestFn/invoke

interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn

main.clj:  437  clojure.main/repl/read-eval-print/fn

main.clj:  437  clojure.main/repl/read-eval-print

main.clj:  458  clojure.main/repl/fn

main.clj:  458  clojure.main/repl

main.clj:  368  clojure.main/repl

RestFn.java: 1523  clojure.lang.RestFn/invoke

interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate

interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate

interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn

AFn.java:   22  clojure.lang.AFn/run

session.clj:  218  nrepl.middleware.session/session-exec/main-loop/fn

session.clj:  217  nrepl.middleware.session/session-exec/main-loop

AFn.java:   22  clojure.lang.AFn/run

Thread.java: 1583  java.lang.Thread/run

[borkdude]

@teodorlu's problem seems solved with the new analyzer (at least the branch I have ready in PR #751 but perhaps works on main too.

[borkdude]

@zampino's issue also seems solved.