Few indexes are not indexed after upgrading to 9.3.2 from 6.3.8 (ALTER REBUILD KNN)

[mohdmsl]

Bug Description:

Few tables are not indexed with below error

[Tue May 27 13:10:27.393 2025] [3697772] WARNING: table 'tag_20250328_2121': disabled at the JSON config
[Tue May 27 13:10:27.393 2025] [3697772] WARNING: table 'tag_20250328_2121': prealloc: disk chunk /var/lib/data/manticore/tag_20250328_2121/tag_20250328_2121.0: prealloc failed: Unable to load KNN index: /var/lib/data/manticore/tag_20250328_2121/tag_20250328_2121.0.spknn is v.1, binary is v.2 - NOT SERVING
[Tue May 27 13:10:27.397 2025] [3697772] WARNING: table 'kbentity_20250512_4278': table 'kbentity_20250512_4278': morphology option changed from config has no effect, ignoring
[Tue May 27 13:10:27.397 2025] [3697772] WARNING: table 'kbentity_20250512_4278': disabled at the JSON config
[Tue May 27 13:10:27.397 2025] [3697772] WARNING: table 'kbentity_20250512_4278': prealloc: disk chunk /var/lib/data/manticore/kbentity_20250512_4278/kbentity_20250512_4278.0: prealloc failed: Unable to load KNN index: /var/lib/data/manticore/kbentity_20250512_4278/kbentity_20250512_4278.0.spknn is v.1, binary is v.2 - NOT SERVING

manticore component versions are:

+-----------+----------------------------------+
| Component | Version                          |
+-----------+----------------------------------+
| Daemon    | 9.3.2 428075261@25050208         |
| Columnar  | columnar 4.2.1 0fe99f4@25050206  |
| Secondary | secondary 4.2.1 0fe99f4@25050206 |
| KNN       | knn 4.2.1 0fe99f4@25050206       |
| Buddy     | buddy v3.27.3+25050214-c52f3477  |
+-----------+----------------------------------+
5 rows in set (0.00 sec)

Manticore Search Version:

9.3.2

Operating System Version:

linux

Have you tried the latest development version?

No

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed

[mohdmsl]

Is there some change in underlying structure of knn ? as only those tables have not been indexed which has vector field

[sanikolaev]

Hi @mohdmsl,

Is there some change in underlying structure of knn ?

Yes. We're thinking about adding support for ALTER TABLE ... REBUILD KNN for cases like this. Unfortunately, since KNN is still a new feature in Manticore and its format is evolving, there's no backward compatibility yet. However, we're also considering adding compatibility for future versions.

[mohdmsl]

Thanks @sanikolaev when can we expect support for ALTER TABLE ... REBUILD KNN ? as rebuidling these indexes from scratch in our environment is quite painful.

[sanikolaev]

when can we expect support for ALTER TABLE ... REBUILD KNN

At this point, we can't commit to a specific timeline for when it will be available. We'll be sure to share updates as soon as we have more concrete plans or when it's done. Appreciate your patience and feedback!
If it's mission critical to be done ASAP pls consider using our professional services - https://manticoresearch.com/services/

[mohdmsl]

Thanks @sanikolaev

[github-actions]

Thanks for the update! However, the internal checklist is not yet complete. A Manticore team member will close this issue once everything is checked off. 🛠️