M: I need to read the exact codec settings from an mov file using ffmpeg, then use those settings to turn a png file into a 5 second mov file using those settings, for the purpose of concatenation. Can you help with this.

G: To achieve this, you'll follow these steps using FFmpeg:

Extract the codec settings from the source .mov file: Use FFmpeg's ffprobe tool to read the codec settings of the .mov file.
Create the .mov file from the .png using the extracted settings: Use the extracted settings to encode your PNG into a .mov file.
Ensure compatibility for concatenation: Ensure the output file is compatible with the original .mov for seamless concatenation.

Step 1: Extract Codec Settings

Run the following command to get codec information:

ffprobe -v error -select_streams v:0 -show_entries stream=codec_name,codec_tag_string,width,height,r_frame_rate,pix_fmt -of default=nw=1:nk=1 input.mov

This will give you:

codec_name: The codec (e.g., prores, h264, etc.).
codec_tag_string: Codec tag string.
width and height: Resolution.
r_frame_rate: Frame rate.
pix_fmt: Pixel format.

Step 2: Create a `.mov` from the `.png`

Use the extracted settings to encode the .png file into a 5-second .mov. For example, suppose the settings are:

Codec: prores
Resolution: 1920x1080
Frame rate: 30 fps
Pixel format: yuv422p10le

Run the following command:

ffmpeg -loop 1 -i input.png -c:v prores -pix_fmt yuv422p10le -s 1920x1080 -r 30 -t 5 output.mov

Explanation:

-loop 1: Loops the PNG to create a video.
-c:v prores: Sets the codec to prores.
-pix_fmt yuv422p10le: Matches the pixel format.
-s 1920x1080: Sets the resolution.
-r 30: Sets the frame rate.
-t 5: Specifies a 5-second duration.

Step 3: Concatenate the Files

Concatenate the files using FFmpeg's concat demuxer. First, create a text file file_list.txt with the following content:

file 'input.mov'
file 'output.mov'

Then run:

ffmpeg -f concat -safe 0 -i file_list.txt -c copy concatenated.mov

This will combine input.mov and output.mov without re-encoding, preserving quality.

If you encounter specific issues (e.g., codec mismatch), let me know, and we can adjust the process further!

M: What if probe reports pix_fmt as (base)?

G: If ffprobe reports pix_fmt as (base), it typically means the pixel format is not explicitly reported or may be interpreted differently by FFmpeg. In such cases, FFmpeg defaults to the "base" pixel format for the codec used (e.g., yuv420p for H.264).

To handle this:

Identify the Default Pixel Format for the Codec:
- Check the codec (e.g., prores, h264) reported by ffprobe.
- Look up the typical pixel format for that codec. For example:
  - H.264 commonly uses yuv420p.
  - ProRes might use yuv422p10le or yuv444p10le.
Test with Common Pixel Formats: If you're unsure, you can experiment with common formats like yuv420p, yuv422p, or yuv444p. Most codecs will reject incompatible pixel formats, so FFmpeg will notify you if you choose the wrong one.
Set Pixel Format Explicitly: In your FFmpeg encoding command, include the -pix_fmt option with a standard format (e.g., yuv420p):

ffmpeg -loop 1 -i input.png -c:v h264 -pix_fmt yuv420p -s 1920x1080 -r 30 -t 5 output.mov

Leave -pix_fmt Out if Compatible: If your initial .mov is straightforward (e.g., H.264 with yuv420p), omitting the -pix_fmt flag might work because FFmpeg will use the default pixel format for the codec:

ffmpeg -loop 1 -i input.png -c:v h264 -s 1920x1080 -r 30 -t 5 output.mov

Inspect Original File for Exact Pixel Format: Run the following to attempt more detailed inspection of the original file:

ffprobe -v error -show_streams input.mov

Look for pix_fmt under the video stream. If still unclear, try ffmpeg -i input.mov for additional hints.

If you're still unsure or encounter errors, share the codec name and other details extracted from ffprobe, and I can refine the steps further!

M: Please repeat those instructions, incorporating the fact that the original mov has an audio stream

G: If the original .mov file includes an audio stream, you need to handle both the video and audio streams to ensure compatibility for concatenation. Here’s how to adjust the process:

Step 1: Extract Codec Settings (Including Audio)

Run ffprobe to gather codec details for both video and audio:

ffprobe -v error -show_streams -select_streams v:0 input.mov

This extracts video settings (e.g., codec_name, width, height, r_frame_rate, pix_fmt). Then, run for audio:

ffprobe -v error -show_streams -select_streams a:0 input.mov

Look for:

Video codec (e.g., h264, prores).
Pixel format (if unclear, default to a common format like yuv420p for H.264).
Resolution (width and height).
Frame rate (r_frame_rate).
Audio codec (e.g., aac).
Audio sample rate (e.g., 44100 Hz or 48000 Hz).
Channel layout (e.g., stereo).

Step 2: Create a Video with the Same Audio Stream

You can include the original audio stream in your new file or add silence. Here's how:

Option 1: Include Silence

If you don't have an audio source, generate silence matching the original audio properties:

ffmpeg -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 \
-loop 1 -i input.png -c:v h264 -pix_fmt yuv420p -s 1920x1080 -r 30 \
-t 5 -shortest -c:a aac -ar 44100 -b:a 128k output.mov

Option 2: Use the Original Audio Stream

Extract the audio from the original .mov file and add it to your new .mov:

Extract Audio:

ffmpeg -i input.mov -vn -c:a copy audio.m4a

Create the Video with Audio:

ffmpeg -loop 1 -i input.png -i audio.m4a -c:v h264 -pix_fmt yuv420p \
-s 1920x1080 -r 30 -t 5 -shortest -c:a aac output.mov

Step 3: Concatenate the Files

Prepare a file_list.txt with the following content:

file 'input.mov'
file 'output.mov'

Then concatenate the files:

ffmpeg -f concat -safe 0 -i file_list.txt -c copy concatenated.mov

Notes:

Audio Stream Matching: Ensure the audio codec, sample rate, and channel layout match the original file.
Pixel Format: If pix_fmt is unclear, default to yuv420p for H.264 or yuv422p for ProRes. Test compatibility.
Duration Alignment: Ensure both video and audio streams in output.mov are exactly 5 seconds.

If you run into issues (e.g., mismatched codecs, incompatible audio), share details, and I can help troubleshoot further!

M: The concatenation keeps throwing “[vost#0:0/copy @ 0x55b137943600] non-monotonic dts “ warnings at me, and the merged file only shows the first file.

G: The non-monotonic DTS error and the issue of the concatenated file only showing the first input occur when the timestamps of the video streams in the concatenated files are not continuous or compatible. To resolve this issue, ensure that all input files have properly aligned and compatible streams for concatenation.

Here’s how to fix it:

Step 1: Use the Concat Demuxer (Requires Re-encoding)

When files aren't natively compatible, you must re-encode them to ensure consistent codec, resolution, frame rate, pixel format, and audio properties. Follow these steps:

Prepare Each File for Concatenation: Re-encode both the original .mov and the new .mov created from the PNG to ensure compatibility. Use the following command for each file:

ffmpeg -i input.mov -c:v libx264 -pix_fmt yuv420p -r 30 -c:a aac -ar 44100 -ac 2 reencoded_input.mov

For the PNG-based file:

ffmpeg -loop 1 -i input.png -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 \
-c:v libx264 -pix_fmt yuv420p -r 30 -t 5 -c:a aac -shortest reencoded_output.mov

Create a File List for Concatenation: Create a text file file_list.txt with the following content:

file 'reencoded_input.mov'
file 'reencoded_output.mov'

Concatenate the Files: Run the concatenation command:

ffmpeg -f concat -safe 0 -i file_list.txt -c copy concatenated.mov

Step 2: Debug the Timestamps

If you need to debug timestamp issues in the original files, you can inspect the DTS/PTS values using:

ffprobe -show_packets -select_streams v:0 input.mov

This will let you verify whether there are inconsistencies in the timestamps of the original files.

Step 3: Use the Concat Protocol for Simpler Cases

If re-encoding isn't desired and the input files are supposed to be compatible (same codec, resolution, frame rate, pixel format, and audio), you can use the concat protocol. However, this requires binary compatibility:

Convert Inputs to a Compatible Raw Stream: Run:

ffmpeg -i input.mov -c copy -bsf:v h264_mp4toannexb -f mpegts intermediate1.ts
ffmpeg -i output.mov -c copy -bsf:v h264_mp4toannexb -f mpegts intermediate2.ts

Concatenate the Streams: Then, concatenate them:

ffmpeg -i "concat:intermediate1.ts|intermediate2.ts" -c copy -bsf:a aac_adtstoasc concatenated.mov

Additional Tips

Ensure Compatibility: All files must match in codec, frame rate, resolution, pixel format, and audio codec/sample rate.
Force Duration for Static PNGs: If the PNG-based video duration is causing issues, use -t explicitly to set the duration.
Validate Inputs: Always check inputs with ffprobe to ensure properties align.

Let me know if any step isn’t clear or if further troubleshooting is needed!

M: Okay, in comparing the ffprobe outputs, the audio streams are the same, while the video streams are thus: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080, 10549 kb/s, 60 fps, 60 tbr, 60k tbn (default) Video: h264 (High) (avc1 /0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 372 kb/s, 60 fps, 60 tbr, 15360 tbn (default) Can you tell me where the incompatibility is occurring based on that?

G: The incompatibility likely arises from differences in the bitrate and the timebase (tbn) between the two video streams:

Bitrate:
- First file: 10549 kb/s
- Second file: 372 kb/s
- These differences are unlikely to cause direct concatenation errors but could lead to playback or quality inconsistencies.
Timebase (tbn):
- First file: 60k tbn (60,000 ticks per second)
- Second file: 15360 tbn (15,360 ticks per second)
- This is a major issue because FFmpeg uses the tbn to calculate timestamps, and mismatched timebases cause DTS/PTS inconsistencies (non-monotonic dts warnings).

Fixing the Incompatibility

You need to standardize the tbn, bitrate, and other settings between the two files. Follow these steps:

Step 1: Re-encode the PNG-Based Video with Matching Properties

Re-encode the PNG-based .mov file to align with the properties of the first .mov file (the higher-quality source). Use the following FFmpeg command:

ffmpeg -loop 1 -i input.png -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 \
-c:v h264 -profile:v high -pix_fmt yuv420p -s 1920x1080 -b:v 10549k -r 60 -t 5 \
-timebase 1/60000 -c:a aac -ar 44100 -ac 2 reencoded_output.mov

Explanation:

-b:v 10549k: Matches the video bitrate of the first file.
-r 60: Matches the frame rate of the first file.
-timebase 1/60000: Matches the tbn of the first file.
-c:a aac -ar 44100 -ac 2: Ensures audio compatibility.

Step 2: Re-encode the Original `.mov` File (Optional)

If the original .mov file isn't already optimized for concatenation, you can re-encode it with these settings to ensure full compatibility:

ffmpeg -i input.mov -c:v h264 -profile:v high -pix_fmt yuv420p -s 1920x1080 -b:v 10549k -r 60 \
-timebase 1/60000 -c:a aac -ar 44100 -ac 2 reencoded_input.mov

Step 3: Concatenate the Re-encoded Files

Create a file_list.txt for the re-encoded files:

file 'reencoded_input.mov'
file 'reencoded_output.mov'

Then concatenate them:

ffmpeg -f concat -safe 0 -i file_list.txt -c copy concatenated.mov

Why This Works

By standardizing:

Bitrate: Both videos will now have the same compression settings.
Timebase (tbn): Aligning the timebase prevents timestamp inconsistencies.
Frame Rate and Resolution: Matching these ensures playback consistency.
Audio Settings: Ensuring the audio stream matches avoids sync or codec issues.

Let me know if you encounter further issues!

Step 1: Extract Codec Settings

Step 2: Create a .mov from the .png

Step 3: Concatenate the Files

Step 1: Extract Codec Settings (Including Audio)

Step 2: Create a Video with the Same Audio Stream

Option 1: Include Silence

Option 2: Use the Original Audio Stream

Step 3: Concatenate the Files

Notes:

Step 1: Use the Concat Demuxer (Requires Re-encoding)

Step 2: Debug the Timestamps

Step 3: Use the Concat Protocol for Simpler Cases

Additional Tips

Fixing the Incompatibility

Step 1: Re-encode the PNG-Based Video with Matching Properties

Step 2: Re-encode the Original .mov File (Optional)

Step 3: Concatenate the Re-encoded Files

Why This Works

Step 2: Create a `.mov` from the `.png`

Step 2: Re-encode the Original `.mov` File (Optional)