Mega 14,000 pic P2E findings

[think(box)]

Greetings to all - I am a new forum member, although I have been using

PTE origination software and P2E shows for several months. I will use those

two terms to denote the software we get from Wnsoft.com web site and the

software we create with it, respectively. It shouldn't be too confusing.

PTE is the greatest - my thanks go out to everyone on the team and all of

the users who have contributed feedback along the way! The P2E shows

are out of this world cool.

I have been pushing the limits of PTE and would like to share findings about

various mega shows that I have created and tested. In this lengthy (sorry)

topic note I provide show size info, show design feature descriptions, PTE

origination software and P2E show play software performance findings, and

info about bugs and limitations encountered in PTE V3.80 and PTE V4.01

in a section labeled to be for people who know enough to be dangerous :-)

Here is some size and feature info for my larger shows:

* 5,000 to 14,000 pictures

* Photo set galleries with from 200 to 400 clickable index thumbnails

* Clickable screen after each photo set for returning to current gallery photo

* Music that is started from the photo show by a button click (a second PTE "show")

* Mega photo show file size from 500MB to 1.1GB (<700MB when made for CD)

* CD-based small P2E show with brief music for mega show install/remove/CD-play

* Most shows start in 6 to 25 seconds, even a 14,000 photo show (400MHz+)

* System resources required for even a 1.1GB 14,000 pic show are very small;

tested on a 150MHz P1 CPU with 72MB RAM: 1. It works! 2. Starts in 60 seconds!

* PTE-based P2E show creation varies from 6 to 25 minutes "create" time on

400 and 650MHz desktop and notebook systems, resp.

Sample show content, organization and distribution:

Years of family 35mm slides scanned and organized into an indexed mega show

complete with separate P2E music player for independent control and no-wait

startups. An all-Windows autorun CD-ROM runs a small P2E one-screen show

(built-in pleasant music on this small, installer show) that gives users buttons

for install, view from CD only, remove, help and exit. The install option copies

showfiles, puts all-Windows shortcuts in the Programs menu and runs it, all from

a single user button click.

The mega show itself has home screen buttons for start music, start gallery,

start show, instructions and exit. The gallery is many screens of thumbnails

linked to hundreds of photo sets in a several thousand photo show. Between

every pair of photo sets is a clickable photo that tells the user to click to return

to gallery (if they wish). This links back to the just-watched photo set clickable

gallery photo so that the user can click again to watch the same set once more,

or they can move around from where they left off in the gallery.

The mega show has NO music. The "start music" button in the mega show does

a P2E "start external application and continue" button function. The external

application is a P2E music player. It is a P2E show exclusively for music playback.

The track artist and selection are provided in text in a fixed-size window that

has P2E comment (filename) used for track info, a P2E control pad for music play

control (pause is actually a repeat track feature), and an "Exit Player" button

that runs a tiny cleanup application since I have found that P2E and the Windows

Media Player sometimes leave the MP3 music files behind in Windows temp area.

Each P2E "photo" is a blank background color "photo" that is programmed to

display for the length of time that the music track lasts. Hence music playback

is timed for each "track".

Separating music from photo set show allows the user to start photo show very

rapidy, choose music only when they want to, control music separately, start

watching show while music "starts" (loads two hours of MP3s in WMP), and even

to play just music if they want. It also makes it possible for me to help them keep

junk clutter cleaned up in the Windows temp area - as part of player exiting.

The show install CD runs on any Windows platform and has all three P2E .exe

files, a self-extracting zip .exe for windows menu loading, and a few loading,

playing from CD, removing, etc., related applications. You can play the show,

even music, directly from the CD-ROM without "installing" anything. Even the

music area cleanup works from CD-based show playback. There are no lags

in performance when played from CD. And the install option completes in no

more time than it takes to read the whole CD at full reader speed. No extra

copies of the huge show .exe file are made on the boot drive, even temporarily.

Show USERS are my extended family members, many of whom are in my opinion

barely qualified to be owning and operating a PC. Therefore everything is easy,

automatic, and suitable for "dummies". Not at all meant to be derogatory, this

is the best way to design applications!

I have made the show in "template" form so that I can make add-ons and whole

new shows with the greatest of ease.

Now for one feature comment - Gallery vs. "Slider control":

Another user has requested a "slider control" to have random access to large

shows. That would certainly be nice. A linked gallery on the other hand takes

some effort to create, yet is best for the viewer. A slider control built into P2E

would be most welcomed though. Please consider it!

Template show generation method (doesn't take very long for a new show!):

Starting from a PTE-made template .PTE file, I use an emacs keyboard macro that

determines link slide numbers by simply reading from PTE-placed photo info in the

PTE file being edited, and then placing photo numbers in appropriate PTE action

line arguments. The gallery photo filenames are generated as part of the macro,

from info already in the PTE file. I use variations on these themes too.

It takes about five minutes for me to reenter the (documented) emacs macro for a

new show and about 30 seconds for emacs to complete the PTE edits for an 8,000

photo show with gallery.

It takes PTE from 6 to 25 minutes to generate the show on a 650MHz P3 notebook

with 128MB of RAM. Best performance is with all pic's on external 1394 or USB drive.

System resource needs:

I would like to differ in opinion with others who have stated the belief that system

resource needs are large and not practical in mega shows. To the contrary, I am

super-impressed with what the PTE/P2E developers have accomplished. Hats off!

Here are my specific findings for a worst-case show tested so far, with 14,000 photos

and a show file size of 1.1GB. Expert users - you'll like what you read, yet developers

please note that there is some important application feedback herein!

Show format: 14,000 pictures, random playback order, no gallery, music start button

in home screen that you get from "Home" keyboard key, that is linked to second P2E

show for music as described above. The 1.1GB P2E showfile has no music. The P2E

music show file is about 62MB (90 minutes at 128Kbps) of MP3 music.

To RUN the PTE V3.80-COMPILED P2E .EXE SHOW FILE:

I took the slowest, oldest, junkiest machine I could find and recorded performance

metrics. That machine is a 150MHz Pentium 1 overdrive with a paltry 72MB of RAM.

Internal bus transfer speed is a pathetic 2MB/sec from hard drive. [My 400MHz P2

system, by comparison, achieves 15-22MB/s disk transfer speeds through PCI IDE.]

Back to the slow machine description: OS software is Win98 Second Edition with

every update available, including IE6. Graphics support is a pathetic 1MB video

RAM and is built in to the mainboard.

It only gets better than what I measured with that 8 year old clunker machine

for the most part.

Show P2E file startup is only 60 seconds on that old machine. This is a 14,000 pic

show that has a 1.1GB file size! Show playback is smooth, even though I used a

100ms simple fade from picture to picture, with 4 second display time per pic. The

show plays back this well whether created with V3.80 or V4.01 of PTE software.

Startup times differ.

SYSTEM EXPERT INFO:

Please stop here if you don't know and/or don't want to know what goes on

under the hood of your machine.

System resources as reported by System Tools --> Resource Monitor (Win98 app):

Metric /// Before starting show /// After starting show /// (then when running)

Allocated Memory /// 161MB /// 172MB ///(180MB running)

Disk Cache /// 38.5MB /// 22.3MB while running

Unused Phys Memory /// 2.3MB /// ~zero /// ~zero

Swap file size /// 100MB /// 104MB /// (122MB running)

Swap file size in use /// 40MB /// 40MB /// (35MB running)

Bytes READ in file system during show startup /// 3 to 5MB /// (50KB/s running)

(from the 1.1GB show file)

CPU kernel utilization /// 12% (for Res. Monitor) /// 100% for 1 minute /// (and CPU is

at 50% when show is running at 4sec/pic, 100ms fade & Resource Monitor is running)

Page faults /// zero /// sporadic /// (~300 running)

In summary, a mega show at 50% of PTE's application limits (spec'd at 2GB file

size and 32,000 photos max) uses around 18MB to 19MB of RAM. It does not load

the 1.1GB image into the swap file as some have suggested may happen, and it

runs very efficiently using about 40% of a 150MHz P1 machine with 72MB RAM.

This means a 60MHz Pentium clunker could just about keep up with resource needs

of this 14,000 picture mega show.

Bug findings on BOTH V3.80 and V4.01 except as noted:

When P2E file exceeds 1GB, any Win98SE system will abort P2E creation with an

"Out of memory" error, even with 512MB main memory and >2.5GB free space for

swap and temporary files on C: drive, with ALL show-related files stored on another

physical hard drive than that used for C: drive. This happens whether or not an

icon was specified for the P2E output .exe file. Except that the icon does not make

it into the .exe file when show creation aborts, the show that is created is actually

complete and works correctly. Apparently since 100% was reached in creation, the

.exe file is complete and operable.

When you do NOT specify an icon in V4.01, the "Out of memory" abort still happens

after reaching 100% creation, yet the snazzy new default icon DOES make it into

the .exe file. There was no default icon in V3.80.

I will second the finding of another user that when you specify an icon (and the

show file is <1GB....) the show creation time doubles, as the .exe file appears to

shuffle onto the boot drive and back to the storage drive *after* hitting 100%.

This is the same in both software versions.

Music and show "link" buttons slow down the show startup. Music startup delay

was definitely reduced in V4.01, when compared to that of V3.80. But unfortunately

show startup was dramatically slowed down in V4.01, to the extent of being not

practical in mega shows. I am keeping V3.80 around on my notebook system,

with v4.01 on desktop system, until this is cleared up. First I will explain what

happens on a show without photo link buttons, then later on - with them.

Here is what happened to the above-mentioned mega show when played on my

400MHz P2 desktop system tested separately with 64MB and 512MB main memory

in order to show that there is little performance dependency:

V3.80-created .exe file startup time was 26 seconds (14,000 picture show)

V4.01-created .exe file startup time was 12.5 minutes (14,000 picture show)

More modest shows of 5,000 pictures still exhibit a similarly large slow-down factor

in show startup. And as other users have pointed out there is no status during

this time. You might expect that the app would never start if you shut the system

down in less than 12 minutes after trying to start P2E show.

I studied the performance during startup for both V3.80 and V4.01 and found that

the rate of page faulting is high and short in duration for V3.80-created .exe files,

while the rate of page faulting is slow and exponentially decreasing in rate for

V4.01-created shows. CPU utilization stays at 100% during this startup activity

in both cases. The final amount of memory used is the same, 18-19MB, in both

cases once the startup as completed.

I/O reads are sporadic and low overall in both PTE versions, with only about

3-5MB total read-in from that huge 1.1GB image. But in V4.01-created shows the

read-in rate also exponentially slows down to just kilobytes per second as the

slow startup proceeds. A lot of processor cycles are spent on little accomplishment.

Performance WITH photo number link buttons or clickable goto-link photos:

The very existence of these really important features causes show startup to be

slowed down substantially, in both V3.80 and V4.01-created shows. Yet in V4.01

the startup time is 5-10X longer. matching experience without link photos in show.

Further, the show startup time is somewhat proportional to the number of clickable

photos or links in the show. A same-sized show with 400 clickable photo gallery

takes more than twice as long to start (~5 minutes@400MHz) as one with 200

clickable photo gallery (~1-2 minutes@400MHz). In both cases about 5,000 photos

were in show.

I would like to make one final point - if there is any reasonable way to make the GUI

hold a change journal and only update the full data structure when writing the .PTE

save file, this could make the GUI usably fast for making mega shows and better on

modestly-long shows, of course. I know this gets deep into the implementation and

I trust your judgement and respect your constraints.

Thanks and cheers!

Bill

[LumenLux]

Where does "awe" end?

I just came from an orchestra/vocal contata that left me in awe. Then I check in to the comfort of our PTE forum, and low and behold ... .another level of "awe"!

Thank you Bill for letting us in on such a gargantuan project! Not just the Mega Show, but the mega effort you have performed. Talk about organized thought! Reaaaaly, I ran out of superlatives for admiration probably in the first half of your description. I hope you will keep in touch here in the forum. I suspect your input will evoke some thought from many of us.

Thanks again.

[ronwil]

Sorry Bill, my brain ranout of RAM three quarters of the way through. I think I will wait for the movie. Nevertheless congratulations on such dedication and best wishes.

Ron [uK]

[Danabw]

Amazing...how's your day job going? :-)

It would be great to see some screen shots of what you've done (uploading is out of the question!).

Care to post any? (www.beechbrook.com/pte)

Dana

[think(box)]

Thanks for the kind words.... And by day I co-develop kick-butt computers!

Screenshots? I'll try to capture some. I can think of a few notables, like the

fixed-size P2E music player, install and show screens, and my most fancy

gallery screen format from Photoshop's "Contact Sheet II" with some style

enhancements. My simplest gallery format is just a screenful of thumbnails

that abut, macro-created from a template screen that is full of blanks.

A mini P2E show could display everything as it really is, just a lot less of it.

Even a shortened-list music player could be incorporated with the startup

feature - a home screen button click from photo set show.

I never even thought about *posting* any show stuff. Thanks for the request.

Cheers,

[Igor]

Dear Bill,

I'm very pleased to see your wonderful exploration!

It also will help us to make PicturesToExe better and faster when we learn about wishes and needs of the creators of presentations.

* A presentation should always clear the temp folder when it closes normally.

Anyway it won't be actual in the future, because now we're developing the new mp3 player which plays music directly from .exe file. And *no* temp files are required. And presentation with mp3 music will start as quickly as it was without mp3 music.

* Sorry, I'm missed suggestion about "Slider control". Please couldn't someone let me know URL of those necessary topics?

* I'll check that problem with producing phase of a 1GB's .EXE file. Thanks!

* I was wondered that v3.80's presentations start in 50 times faster in comparison with presentations created in v4.0x! Because only since v4.00 we have vastly optimized startup phase. And presentation with large .pte project file (14,000 slides and/or thousands of objects)

starts faster as never before. For example "Jukebox" presentations (165 slides and 50 objects per each slide) produced in v3.80 starts under Athlon 1600MHz in 28 seconds and with optimized v4.00 startup takes only 1/3 of second!

Please couldn't you check again to test v3.80 and v4.00?

Thank you,

And with best regards!

[alrobin]

* Sorry, I'm missed suggestion about "Slider control". Please couldn't someone let me know URL of those necessary topics?

Igor, here is the reference.

http://www.picturestoexe.com/forums//index...ct=ST&f=2&t=573

Hope this is the one you were referring to.

[think(box)]

Greetings Igor,

Wow - that change to the MP3 player to avoid temp usage is perfect! I know that those files are supposed

to be removed, and most of the time they are today. They get left behind when the system has a Windows

or app failure and the normal exit can't happen. When I run a 2-hour music player, that's 100MB in temp.

Thanks Al for the "slider" link. That's the one I discussed. I have thought about how this might be done and

offer the following. A slider is no more than a horizontal scrollbar that may be placed along the bottom edge

of the P2E navigation pad. When the slider is moved to 50% of the scrollbar length the picture will advance

to the half-way point picture count. Since it takes some fraction of a second to read and display a photo it

may not be good to start until the user releases the mouse button. Alternatively you could read next photo

data immediately and create a tiny, low-resource, quick photo preview of where the slider is until the user

releases the mouse button. This tiny preview photo visible during scrolling could be about an inch or 3cm in

size (varies with screen area setting of course) and could be located just below or above the navigation bar

depending on where it has been placed. If we make a navigation bar with less than the full button set, then

the navigation bar is more narrow. I almost always use a single button nav bar - "Pause" - in top right of

screen. This has bearing on the picture scrollbar horizontal pixel dimension. To solve this I suggest that the

entire scollbar vanishes while the show is playing and appears only while the show is paused. You could then

make the scrollbar a fixed, large pixel width - up to the full screen width for ease of use.

By the way, you could make the entire navigation bar collapse to just small "Navigate" and "Pause" buttons

next to each other when the show is running, and expand it to all creator-defined buttons and features only

when "Navigate" is pressed. Show could auto-pause when "Navigate" is pressed, for ease of use. You need

the "Pause" button in that set of two to allow pausing to view the picture without controls all over the place.

The "Pause" keyboard key should do the same as clickable "Pause" button, as it does today.

The above discussion is in reference to the April 16 post that Math made. Then on April 24 Math and Guido

explained the use of "Bride", "Groom", etc. labeled buttons that go away after first slide. You could, of course

make those buttons apply to all slides, but I offer this alternative. Lots of buttons mask my view of the photos

so I would prefer that they appear only when the show is paused as described above. You could, as optional

addition to the horizontal picture scrollbar make a set of delineated "buttons" across another screen-wide

horizontal stripe similar in size and adjacent to the scrollbar. These buttons have text that we add when we

make the show, and a picture number is coded for goto also at show creation time. Button count can be set

to ten maximum, fixed at one-tenth of screen area pixel width, and centered horizontally when displayed,

where less than ten buttons have been defined. I would prefer not to have to precisely place these buttons

in an object editor interface. They can be autoplaced and sized after I define them in a "Project Options" tab

where all I enter is button text, font info and "goto" number for as many buttons needed, up to ten max. For

a slick alternative you could have a GIF button graphic optional user spec for each button. PTE places text

on show creator's GIF button graphic. If you don't define a graphic for each button you get standard buttons.

Igor, for large show startup time debug I think you can make a large debug show by simply adding a modest

number of pictures several times until the count is large. I'll test and let you know what I find if you want.

To make a debug show with gallery, I would try making one slide in object editor with dozens of copied

thumbnails, each having a "goto picture number" definition. That slide could then be copied several times

to make a large debug gallery. The number of "goto" items definitely affects both V3.80 and V4.01 startup

adversely. Since I have some blank gallery templates I can test debug show creation if that would be helpful.

You are correct that show startup got better for small shows, so I agree it is unexpected that it would get far

slower for large shows. As my prior debug info describes, for only V4.01-created shows Windows spends a

huge amount of compute time paging very little, and it is filling no more or less of an image in memory. It

definitely can be 10X faster, as when same large show is made with V3.80. The slowness is quite visible for

shows with well under 14,000 photos. I would make a debug show with more like 3,000 to 5,000 photos.

Please let me know if it would help that I make a debug show for you. It may be possible to do this with a

rather small amount of unique test photo material and a .PTE debug file that you could use to create big show.

Cheers!

[think(box)]

Please excuse me for not giving correct credit to all forum member names in the "slider" picture scrollbar

dialog. That dialog involved Math (Mattias), Al, Guido, Jim and "cc". Jim is the one who mentioned the

"Bride" and "Groom" buttons he uses.

Cheers,

[LumenLux]

Regarding the sound handling method of Bill and the new mp3 player being built by Igor -

With such implementation, can the sound track be paused and resumed simultaneously with the photo flow?

[Igor]

Ok, I also will check with similar PC configuration, you've described (P-150).

I remember that Jukebox which has 600 Kb of .pte project file starts on the notebook PII-233 in 3 seconds when produced in v4.00. And more than 2 minutes if v3.80. Anyway, that is really interesting subject for the exploration!

We plan to add slide control, and I hope about v4.10

This suggestion in our TODO list.

[Danabw]

Gonna post any screen shots? Or a small sampler?

Dana

[Igor]

To think(box)

Yes, you was right about the slow startup with 4000's slides presentation.

It still takes a lot of time to be loaded. About 10 seconds on my Athlon XP 1600+. But "JukeBox" presentation which has 16000 lines of .pte file (and same size - 670 KB) starts up in 0.3 of second.

Now we found a reason of it. It happens because first kind of show has 4000 sections on .pte file. One section for every slide ([slide1] Picture=abc <...> [slide4000] Picture=abc) and second ("JukeBox") has only 37 slides and 37 sections in its .pte file, but with a lot number of objects.

So now we able to fix this problem with a presentations which have more than 300..500 slides. It will require to add special optimization. And even 14000 slides will be loaded during short time.

Thank you again for finding of this interesting problem!

With kindest regards,

[LumenLux]

Igor and Bill - congratulations. The "system" works again in incorporating the various talents of you smart people.!

[think(box)]

Wow - you're the greatest Igor! That was a fast discovery and fix. You've made my day. Thank you!

And to Dana, two replies ago, and the many others who've asked - I am making a small show with large show features that should be ready soon. I lost some critical time this past weekend to a bad sector failure in my 700GB redundant data store. Thanks to the redundant storage and some utility software I was able to both fix the bad disk sector on a relatively new 120GB drive and recover my data. I never knew you could sometimes repair bad sectors on a hard drive. By the way I do not use PCI IDE Raid to manage the redundant storage. I don't have the extra PCI slot for that card.

The bottom line is I'll have to post my show later than expected. I've developed a new special effect to debut that provides a view from infinity.

Thanks again to all,

[RayPar]

I lost some critical time this past weekend to a bad sector failure in my 700GB redundant data store.

I am drooling all over my keyboard. Any plans for a Fibre-Channel attached 2TB storage subsystem?

[Danabw]

I am drooling all over my keyboard. Any plans for a Fibre-Channel attached 2TB storage subsystem?  

Heck, I can beat that. I have a really, really big cardboard box in my garage that's almost completely empty! (It's the only space in my garage that isn't 100% stuffed w/junk). Talk about valuable storage! :-)

Dana

[RayPar]

I am drooling all over my keyboard. Any plans for a Fibre-Channel attached 2TB storage subsystem?  

Heck, I can beat that. I have a really, really big cardboard box in my garage that's almost completely empty! (It's the only space in my garage that isn't 100% stuffed w/junk). Talk about valuable storage! :-)

Dana

Dana,

How about a HP storageworks VA7110, or better yet, EVA5000?

[think(box)]

I am drooling all over my keyboard. Any plans for a Fibre-Channel attached 2TB storage subsystem?

Heck, I can beat that. I have a really, really big cardboard box in my garage that's almost completely empty! (It's the only space in my garage that isn't 100% stuffed w/junk). Talk about valuable storage! :-)

OK, make fun of my 700GB data store.... You know, size isn't everything! Maybe some day when I grow up I'll have a PetaByte as you must. Of course if I can figure out how to make boxes of junk worldwide store valuable data I'll be telling Bill Gates what I would like to have him do next. Hmmm, let me see... I'll sign this ((think)box)

[Danabw]

Must be thinly veiled jealousy on my part. I thought I was "over the top" w/my collection of drives (40g, 60g, 60g, 80g, 120g, 180g) but I clearly am falling behind! (One of the 60's and the 80 are for my removable dive bay, and contain backups of all my important data - swap them monthly to have redundant backups and store one at work while the other is at home).

Now if I could just find time to finish that bomb shelter/pool room... ;-)

Dana