Load ply with uv coordinate

[YangHai-1218]

Hi.
I checked the docs, but I can't find the implementation about loading ply objects with UV coordinates.
Does that mean I must use a third party to load ply object first to extract the uv coordinate?
Thanks.

[bottler]

I have never seen a ply file containing UV coordinates / texture coordinates, and PyTorch3D does not support them at the moment. I'd be interested in where you have got such files from. How frequent are they? Is there a single texture image in a separate file?

[YangHai-1218]

Yes, there is a separate texture image to do UV mapping. The file is provided by a public dataset used to do 6D pose estimation. But the dataset has been re-processed. I don't know if the original dataset also contains UV coordinate in the ply file.
The header format is following:

ply
format ascii 1.0
comment VCGLIB generated
comment TextureFile obj_000008.png
element vertex 8268
property float x
property float y
property float z
property float nx
property float ny
property float nz
property float texture_u
property float texture_v
element face 15728
property list uchar int vertex_indices
end_header

[bottler]

If the format is actually used in a public dataset then can you identify it and supply a link? It could be a useful thing for PyTorch3D to support it.

It would not be so hard to make the change. Would you be interested in doing it? If so, I can give some advice.

[YangHai-1218]

Sorry for the late reply.
You can check this link http://ptak.felk.cvut.cz/6DB/public/bop_datasets/. Here are many datasets for the 6D pose estimation task, and the model-related files are under *.models.zip.
Yes, I am interested in doing it. It would be very nice if you could give me some advice.

[bottler]

That's great. The change here is to loading meshes to ply files in a certain case, in particular when there are texture_u and texture_v in the vertex data, and a file named in a comment like comment TextureFile obj_000008.png. The outer level of the change is in the read method of MeshPlyFormat in pytorch3d/io/ply_io.py, where, in this case, we will give the output mesh a TexturesUV. In particular, the members of the TexturesUV should be initialised as follows:

• maps: Add _read_image to the imports from pytorch3d.io.utils and parse the path from data.header.comments and pass it to _read_image(path, path_manager=path_manager, format="RGB"). You probably need to divide the values by 255.0.
• faces_uvs: because these files store the uvs for each vertex rather than for each corner of each face, you don't need the flexibility that faces_uvs provides. So just supply the same faces tensor to the TexturesUV that is already being supplied to the mesh.
• verts_uvs: These need to come from the texture_u and texture_v values parsed from the PLY, and I've added the logic for this. If these data are present, they will be at data.verts_texture_uvs. You need to check if they need scaling/swapping/reversing. (The functions in pytorch3d/vis/texture_vis.py might be helpful with this).

I've made some modifications to ply_io.ply here to read the verts_texture_uvs and make the header comments available, so you can get started. Internally in the file, I changed the results of parsing to be dataclasses instead of big tuples, because they were getting too big.

[YangHai-1218]

Hi, sorry for the delay.
I complete the feature for supporting reading uv and uv map for ply format. Should I open a PR?

[bottler]

Yes please, if you want to share it.

[kjchalup]

Did this get merged / can I close the issue?

[YangHai-1218]

Yes, it is solved. I will close
the issue.