• Nigel Cunningham's avatar
    (Bug1352028) Add a JSON progress bar for bulk operations. · 55a8deb8
    Nigel Cunningham authored
    This patch adds a JSON progress meter (I'll call it that to avoid confusion
    with progress bars) to the bulk uploading of users, groups and group
    memberships and the bulk export and import of users (LEAP), so the user can see
    the progress of the operation and not just the submit button changed to
    'Processing..' and whatever indication their browser gives while waiting for
    The bulk export and import are minor rewrites, replacing the old iframe based
    progress bar and the associated multiple pages and additional template file in
    the case of the bulk export, and the recursive redirect-to-self of the bulk
    To accomplish the display of the progress bar during the operation, we make the
    PHP session be closed (read only) except when changes need to be made. This is
    for the most part a straightforward change in session.php as it's the only
    direct accessor. In other places, we replace direct accessing of the session
    variable ($_SESSION) with use of the session class ($SESSION) so that it can
    reopen the session, make the change and close the session again.
    There is one more aspect to all of this: with previous behaviour, multiple
    requests for the same session would queue, taking the session lock in turn.
    After this patch is applied, they can proceed in parallel, allowing greater
    throughput. There is no additional locking requirement because the issues are
    the same as those already dealt with in allowing multiple PHP threads to
    process requests from different sessions at the same time.
    I have sought to make the progress meter nice and generic, so it can be used in
    the other bulk imports and exports too.
    Paradoxically, these changes don't just make the import seem to be faster, it
    actually is.. at least in the case of users and groups.
    Times for importing 1000 users, groups and memberships, averaged over 3 runs
    each (Wall time, not CPU time - but the relationship is the same).
                    Without Progress     With Progress
    Users                166s               155s
    Groups                85s                78s
    Memberships           20s                19s
    Change-Id: Iec15c57db32c77994edb80c71d65591de51a95e4
    Signed-off-by: default avatarNigel Cunningham <nigelc@catalyst-au.net>