Issue 7153 - 3DGS IO #7191

lumurillo · 2025-03-08T04:52:33Z

Fixes #7153

Bug fix (non-breaking change which fixes an issue): Fixes #
New feature (non-breaking change which adds functionality). Resolves #
Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #

Motivation and Context

Provide IO capabilities with 3DGS support for files like PLY and SPLAT.

Checklist:

I have run python util/check_style.py --apply to apply Open3D code style
to my code.
This PR changes Open3D behavior or adds new functionality.
- Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is
  updated accordingly.
- I have added or updated C++ and / or Python unit tests OR included test
  results (e.g. screenshots or numbers) here.
I will follow up and update the code if CI fails.
For fork PRs, I have selected Allow edits from maintainers.

Notes

Some of the test may fail because we need that the sample files are uploaded to the corresponding repo.

* Include additional 3DGS properties for reading operation * Add validation methods for 3DGS mandatory properties * Read and Write including additional vertex * Add the validation check to the write method * Add a unit test for a valid ReadCloutPoint operation * PLY read to point cloud * PLY write correction * Inline documentation * Add constant defines for magic numbers and attribute names * Add unit tests for reading and writing 3DGS formatted point clouds * Add 3DGS attribute verification to the read operation --------- Co-authored-by: Juan Cruz <juan.cruz.naranjo@intel.com> Co-authored-by: Gomez Rodriguez, Diego <diego.gomez.rodriguez@intel.com>

* Add splat writer * Add splat reader * Splat reader added to PointCloudIO.h * Fix CMakeList.txt * Add test for reading PointCloud from SPLAT format * Fixing Style --------- Co-authored-by: ikun03 <ikunal03@gmail.com>

update-docs · 2025-03-08T04:52:37Z

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

review-notebook-app · 2025-03-08T04:52:38Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

ssheorey

Thanks @lumurillo and team for this very useful contribution!

Naming: Can we replace the use of 3DGS to GaussianSplat everywhere?
This also take care of variants of 3DGS such as 2D Gaussian Splatting. Also splat seems to be the more common / short name for this representation in other software.

Btw: a good way to remember the f_rest size is that it's n^2-1 for order n :-)

PS: Review is WIP: Will review test and docs later.

ssheorey · 2025-03-08T16:23:34Z

cpp/open3d/t/geometry/PointCloud.h

@@ -214,6 +214,41 @@ class PointCloud : public Geometry, public DrawableGeometry {
    /// This is a convenience function.
    bool HasPointNormals() const { return HasPointAttr("normals"); }



I would probably skip these functions - I don't think they will be used much.
Instead, can I recommend:

int GaussianSplatGetSHOrder() const; // Get order of spherical harmonic data present. // f_rest is absent -> return 0; // return 1, 2, 3 if f_rest.shape[1] is (3, 8, 15) respectively. Throw error for invalid shape. bool GaussianSplatReduceSHOrder(int order=0); // Resize f_rest to the given order to compress it.

ssheorey · 2025-03-08T16:47:52Z

cpp/open3d/t/io/file_format/FilePLY.cpp

    // Other attribute.
    return std::make_tuple(name, 1, 0);
 }

+bool Is3DGSPointCloud(const geometry::PointCloud &pointcloud) {


Can we move this to the PointCloud class, since its more generally useful other than IO? Also rename to PointCloud::IsGaussianSplat().
Also as part of this, we should check shape (as you do below), i.e. an invalid Splat is not a Splat (just another point cloud).

ssheorey · 2025-03-08T16:50:21Z

cpp/open3d/t/io/file_format/FilePLY.cpp

@@ -18,6 +18,29 @@
 #include "open3d/utility/Logging.h"
 #include "open3d/utility/ProgressReporters.h"



I think we can simplify this - the attribute names are fixed, and are used in other parts of Open3D as well. We can directly use the strings for the property names instead of defining macros.

ssheorey · 2025-03-08T16:53:24Z

cpp/open3d/t/io/file_format/FilePLY.cpp

+#define SCALE_SUFFIX_INDEX 6
+#define F_DC_SUFFIX_INDEX 5
+#define F_REST_SUFFIX_INDEX 7
+


The latest C++ guidelines prefer constexpr int to preprocessor macros. In fact with C++26 you mostly don't need the preprocessor at all!
With constexpr int, you will need to put them in an anonymous namespace to ensure they don't leak outside this translation unit.

ssheorey · 2025-03-09T00:31:30Z

cpp/open3d/t/io/file_format/FilePLY.cpp

+#define SCALE_SUFFIX_INDEX 6
+#define F_DC_SUFFIX_INDEX 5
+#define F_REST_SUFFIX_INDEX 7
+


The latest C++ guidelines prefer constexpr int to preprocessor macros. In fact, with C++26 you mostly don't need the preprocessor at all!
With constexpr int, you will need to put them in an anonymous namespace to ensure they don't leak outside this translation unit.
PS: Perhaps POSITIONS_ATTR_DIM (etc.) is more appropriate, since we cannot have less than 3 dims for position (or RGB, etc.) in Open3D. 2D is not supported.

ssheorey · 2025-03-09T07:20:17Z

cpp/open3d/t/io/file_format/FileSPLAT.cpp

+    // Convert color to uint8 (scale, clip, and cast)
+    return (color * 255.0).cwiseMin(255.0).cwiseMax(0.0).cast<int>();
+}
+


Use C++ fstreams with exceptions for error handling for faster IO.

ssheorey · 2025-03-09T07:22:05Z

cpp/open3d/t/io/file_format/FileSPLAT.cpp

+        // Positions
+        DISPATCH_DTYPE_TO_TEMPLATE(positions_attr.dtype_, [&]() {
+            const scalar_t *positions_ptr =
+                    static_cast<const scalar_t *>(positions_attr.data_ptr_);


[nit] Alternative:

data_ptr: tmap["positions"].GetDataPtr<scalar_t>()

group_size: tmap["positions"].GetShape(1)

ssheorey · 2025-03-09T07:29:54Z

cpp/open3d/t/io/file_format/FileSPLAT.cpp

+                    rot.cwiseMin(255.0).cwiseMax(0.0).cast<int>();
+
+            for (int idx = 0; idx < 4; ++idx) {
+                splat_write_byte(splat_file, int_to_byte(int_rot[idx]));


The bounds checking inside int_to_byte is unnecessary, since we have clamped the values above. Simpler static_cast should be faster.

ssheorey · 2025-03-09T07:34:07Z

cpp/tests/t/io/PointCloudIO.cpp

@@ -309,16 +359,11 @@ TEST(TPointCloudIO, WritePTSColorConversion1) {

 // Check PTS color boolean to uint8 conversion.
 TEST(TPointCloudIO, WritePTSColorConversion2) {


Is this .pts file IO test deletion accidental?

ssheorey · 2025-03-09T07:35:35Z

cpp/tests/t/io/PointCloudIO.cpp

+    std::string filename_out = utility::filesystem::GetTempDirectoryPath() +
+                               "/test_sample_right_3dgs_format.ply";
+    std::ofstream outfile;
+    outfile.open(filename_out);


We can make this a global variable (anon namespace) since this is used in multiple places.

Also, you can use raw string literals to avoid typing the \n:

R"(ply format ascii 1.0 ...)"

ssheorey

I think we need to modify the unit tests a bit:

ply:

Start with a ply file as a string [Done].
Create tensors for each of the attributes (manually fill them in). See example here.
Write ply string to disk as ply.
Read it back from disk.
Test - all attributes are present, are the right shape and data type as the tensors in step 2.
Test all attribute values match (See example here).

splat:

Start with .splat file as a binary blob - you can get this by converting the .ply file above to .splat with the reference convert.py script (antimatter github). You can convert it to a C array with xxd -i infile.splat outfile.h. If the C array is too big, let me know and we can add it directly to the git repo (with some cmake additions). This step directly proves that antimatter reference splats can be read by Open3D.
Start with the same tensor attributes as in the ply test.
Write it to disk as splat.
Read it back from disk.
Test - all attributes are present, are the right shape and data type as the tensors.
Test all attribute values match (See example here).

This simplifies the tests by using round trip test, no data file is needed for splat,

ssheorey · 2025-03-10T21:07:42Z

cpp/tests/t/io/PointCloudIO.cpp

+    std::string filename_out = utility::filesystem::GetTempDirectoryPath() +
+                               "/test_sample_right_3dgs_format.ply";
+    std::ofstream outfile;
+    outfile.open(filename_out);


Also, you can use raw string literals to avoid typing the \n:

R"(ply format ascii 1.0 ...)"

ssheorey · 2025-03-10T21:43:16Z

docs/jupyter/io/files/point_cloud_sampled_only_table.splat

We don't add data to the repo (as much as possible), so we can delete it from here. Instead, I'll add it to the Open3D downloads site.

lumurillo and others added 3 commits March 7, 2025 12:45

Add first draft of 3DGS documentation

7fd485f

Add splat writer

b5071b9

* Add splat writer * Add splat reader * Splat reader added to PointCloudIO.h * Fix CMakeList.txt * Add test for reading PointCloud from SPLAT format * Fixing Style --------- Co-authored-by: ikun03 <ikunal03@gmail.com>

lumurillo changed the title ~~Issue 7153 dev~~ Issue 7153 - 3DGS IO Mar 8, 2025

ssheorey requested review from benjaminum, ssheorey and aditya18007 March 8, 2025 06:20

ssheorey requested changes Mar 9, 2025

View reviewed changes

ssheorey requested changes Mar 10, 2025

View reviewed changes

ssheorey added the hackathon label Mar 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 7153 - 3DGS IO #7191

Issue 7153 - 3DGS IO #7191

lumurillo commented Mar 8, 2025

update-docs bot commented Mar 8, 2025

review-notebook-app bot commented Mar 8, 2025

ssheorey left a comment •

edited

Loading

ssheorey Mar 8, 2025

ssheorey Mar 8, 2025

ssheorey Mar 8, 2025

ssheorey Mar 8, 2025

ssheorey Mar 9, 2025

ssheorey Mar 9, 2025

ssheorey Mar 9, 2025

ssheorey Mar 9, 2025

ssheorey Mar 9, 2025

ssheorey Mar 9, 2025

ssheorey Mar 10, 2025

ssheorey left a comment

ssheorey Mar 10, 2025

ssheorey Mar 10, 2025

		@@ -214,6 +214,41 @@ class PointCloud : public Geometry, public DrawableGeometry {
		/// This is a convenience function.
		bool HasPointNormals() const { return HasPointAttr("normals"); }

		@@ -18,6 +18,29 @@
		#include "open3d/utility/Logging.h"
		#include "open3d/utility/ProgressReporters.h"

		@@ -309,16 +359,11 @@ TEST(TPointCloudIO, WritePTSColorConversion1) {

		// Check PTS color boolean to uint8 conversion.
		TEST(TPointCloudIO, WritePTSColorConversion2) {

Issue 7153 - 3DGS IO #7191

Are you sure you want to change the base?

Issue 7153 - 3DGS IO #7191

Conversation

lumurillo commented Mar 8, 2025

Motivation and Context

Checklist:

Notes

update-docs bot commented Mar 8, 2025

review-notebook-app bot commented Mar 8, 2025

ssheorey left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ssheorey left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ssheorey left a comment •

edited

Loading