MBS Xojo Conference
6/7th September 2018, Munich

« Import images from ca… | Home | Configure Toolbar Men… »

Tips for FileMaker Data Migration Tool


You may have seen the release of the FileMaker data migration Tool. And you can read documentation here. We got a few extra tips for you here:



First, be aware that matching of tables is by name or ID. So please do not reuse field or tables names. Because due to name matching, if you drop a table with a given name, a few days later add it again for a different reason, the migration could move data from the old table to the new table and match fields just by ID. This can cause serious problems, if data ends up in a different column or table!

e.g. if you have a table test with ID, FirstName, LastName, Job. Than later in development test is deleted. Days after that you create a new table named test with fields Product ID, PersonID, Payment, ID which is totally unrelated to the old table. But now when you migrate, the data from FirstName ends up in PersonID, if those have a matching internal field ID!

Second, please don't use the normal full access account to migrate. Better make a new account with a privilege set where everything is forbidden except the privilege named fmmigration. This privilege name can have an unique add-on, e.g. fmmigrationMySecret and that must match between source and clone files to allow migration. And you want to use that to avoid anyone with an account to just migrate your solution to steal the data.

Third, please be aware that using the normal Terminal window involves using a shell application, which may (or may not) store a permanent history of commands. See man page of history command to clear. This may include passwords. Also the command line may be visible with other tools while running like ps tool. So never ever put your full access password there as the trouble with someone seeing it, is big.

Forth, please check the MBS Shell functions to run the FileMaker Data Migration Tool. MBS plugin can run several shells in parallel to migrate several files parallel and better use CPUs to get the jobs done quicker. Our plugin can return you the messages output by the tool and if you run it directly, no bash is involved, so no history is written. By using our write commands, you can even send text like passwords in memory to the tool. Passing parameters does not need quoting with out shell functions, as you pass each parameter as a parameter to the plugin function.

Here is a sample script:

Set Variable [ $shell ; Value: MBS( "Shell.New" ) ] 

Set Field [ Shell::Output ; "" ] 

Set Field [ Shell::Error ; "" ] 

Commit Records/Requests [ With dialog: Off ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "-src_path") ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "/Users/cs/Desktop/Contacts.fmp12") ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "-clone_path") ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "/Users/cs/Desktop/Contacts Clone.fmp12") ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "-target_path") ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "/Users/cs/Desktop/Contacts target.fmp12") ] 

Set Variable [ $s ; Value: MBS( "Shell.AddArgument"; $shell; "-v") ] 

Set Variable [ $s ; Value: MBS( "Shell.Execute"; $shell; "/Users/cs/Downloads/FMDataMigration") ] 

Set Variable [ $error ; Value: "" ] 

Set Variable [ $result ; Value: "" ] 

If [ MBS("IsError") ] 

    Show Custom Dialog [ "Failed to run" ; $s ] 

Else

    If [ Length(Shell::Input) > 0 ] 

        Set Variable [ $s ; Value: MBS( "Shell.WriteInputText"; $shell; Shell::Input; "UTF-8") ] 

    End If

    # Loop while app runs and collect messages

    Loop

        Set Variable [ $s ; Value: MBS( "Shell.Wait"; $shell; 1) ] 

        Set Variable [ $error ; Value: $error & MBS( "Shell.ReadErrorText"; $shell; "UTF-8") ] 

        Set Variable [ $result ; Value: $result & MBS( "Shell.ReadOutputText"; $shell; "UTF-8") ] 

        Set Field [ Shell::Error ; MBS( "Text.ReplaceNewline"; $error; 1) ] 

        Set Field [ Shell::Output ; MBS( "Text.ReplaceNewline"; $result; 1) ] 

        Exit Loop If [ MBS( "Shell.IsRunning"; $shell) ≠ 1 ] 

    End Loop

    Commit Records/Requests [ With dialog: Off ] 

End If

Set Variable [ $r ; Value: MBS("Shell.Release"; $shell) ] 



Finally, if you build something, be aware that you may not have the right to redistribute the FileMaker Data Migration Tool on your own. FileMaker developers need to have a FDS subscription to be allowed to download it.
09 06 18 - 10:51
ten comments

If I understand correctly, you’re saying the plugin will run the shell command inside itself, effectively. It’s not opening Terminal and running it in there. Is that correct?

Also… Is it possible to have the plugin run the migration tool with the tool stored in a container field or somewhere else, or does the tool have to be installed into the user’s file system? Or… I suppose we can use the plugin to install the tool, and then remove it at the end if desired…?

Thanks!
David Thorp - 24 06 18 - 21:57

The command runs invisible to the user in the background. No terminal window.
The tool must be on disk and be marked executable on Mac & Linux (see Files.SetPosixPermissions).
You can of course export from container field to temp folder and run it.
Christian Schmitz (URL) - 24 06 18 - 23:01

Ok. Thanks.
David Thorp - 25 06 18 - 01:42

Hi again Christian. I’m looking through your example above. It all makes sense, I think, except:

1. I’m having trouble understanding what these lines are for:

If [ Length(Shell::Input) > 0 ]
Set Variable [ $s ; ... ]
End If

I see that we create the shell object add all the arguments, then run the migration tool in the shell object.

Then check for an error with the execute, and if not that, then…

... those lines above relating to Shell::Input. Then…

Loop, collecting messages until the shell isn’t running any more.

Could you explain what those lines about input are for?

2. I would think it makes more sense to the two “Text.ReplaceNewline” set fields after the loop instead of inside it – build up the $error and $result variables, and then put them both into the fields after exiting the loop…? Is there are a reason you chose to put those two lines inside the loop?

Thanks!
David Thorp - 02 08 18 - 06:18

I think I figured out the answer to my question 2. Those two set fields are inside the loop because we want to SEE the messages display in the two fields while the shell is running. If that’s correct then please feel free to ignore question 2.

However, I’m still not clear on the purpose of the Shell::Input lines.

Thanks!
David Thorp - 02 08 18 - 06:54

You don’t send input to the migration tool, so don’t use it.
Except maybe for passing a password?
Christian Schmitz - 02 08 18 - 12:42

Ok so those three lines aren’t really needed here. Sounds good to me. Maybe you want to update the code in your example above (by removing those three lines)? Others may be confused by it too? Although I suppose with these comments here it clears it up for anyone reading, anyway :-)

As always, thanks!
David Thorp - 02 08 18 - 19:20

Some more questions about this process. ;)
I’m having trouble understanding the purpose of the Wait function.
Does that function actually pause the FileMaker script? Does it pause the shell script?
What practical difference does it make if that step isn’t there? ... or instead if the Wait amount is a lot longer?
Is it basically the same as putting a FileMaker Pause/Resume script[1 second] in there instead or is there some significant difference between that and the Wait function?

Hope that makes sense. Thanks!
David Thorp - 13 08 18 - 05:19

You can use Shell.Wait to block FileMaker for a few seconds and it will return as soon as the tool finishes.
This also works with FileMaker Pause Script Step. Just Shell.Wait will block FM, so it doesn’t do anything.
Christian Schmitz (URL) - 13 08 18 - 09:47

Ok, so it’s basically the same as the FileMaker Pause Script Step, but without the little Continue button showing up in the toolbar, and without any ability to interact with FM during the “wait”. It pauses everything in FileMaker completely and allows no interaction at all, during that second, or whatever time, whereas the FileMaker Pause Script step allows you to click other buttons or run other scripts or menu commands during the FileMaker Pause. Shell.Wait doesn’t allow any of that. Is that what you mean by “block FM”. Is all that correct? Thanks!
David Thorp - 13 08 18 - 10:10


  
Remember personal info?

Emoticons / Textile


Notify:
Hide email:

Small print: All html tags except <b> and <i> will be removed from your comment. You can make links by just typing the url or mail-address.