1. 23 Apr, 2015 1 commit
    • Nigel Cunningham's avatar
      (Bug785472) Remove unneeded raise_memory_limit calls · 039564a5
      Nigel Cunningham authored
      On the basis of my profiling and testing over the last week,
      all of the scripts affected by this patch don't need their
      raise_memory_limit calls. Ensure_sanity currently sets the
      memory limit to 128MB, which has been seen to be more than
      sufficient for each of these use cases.
      
      Tests have involved at least 1000 records being imported in
      each case, and sometimes as many as 20,000. I would have
      liked to have tested with some really full profiles (eg
      lots of pictures and content), but am satisfied that there's
      enough margin to cover those cases. In any case, such files
      are often handled using external apps and therefore won't
      be counted toward PHP's memory use anyway. Finally, the
      limit can easily be increased by the user if necessary for
      a particular case.
      
      Change-Id: Ifecc83fd47da51268bae6cbd6960735eb91f9403
      Signed-off-by: default avatarNigel Cunningham <nigelc@catalyst-au.net>
      039564a5
  2. 03 Mar, 2015 1 commit
    • Nigel Cunningham's avatar
      (Bug1352028) Add a JSON progress bar for bulk operations. · 55a8deb8
      Nigel Cunningham authored
      This patch adds a JSON progress meter (I'll call it that to avoid confusion
      with progress bars) to the bulk uploading of users, groups and group
      memberships and the bulk export and import of users (LEAP), so the user can see
      the progress of the operation and not just the submit button changed to
      'Processing..' and whatever indication their browser gives while waiting for
      content.
      
      The bulk export and import are minor rewrites, replacing the old iframe based
      progress bar and the associated multiple pages and additional template file in
      the case of the bulk export, and the recursive redirect-to-self of the bulk
      import.
      
      To accomplish the display of the progress bar during the operation, we make the
      PHP session be closed (read only) except when changes need to be made. This is
      for the most part a straightforward change in session.php as it's the only
      direct accessor. In other places, we replace direct accessing of the session
      variable ($_SESSION) with use of the session class ($SESSION) so that it can
      reopen the session, make the change and close the session again.
      
      There is one more aspect to all of this: with previous behaviour, multiple
      requests for the same session would queue, taking the session lock in turn.
      After this patch is applied, they can proceed in parallel, allowing greater
      throughput. There is no additional locking requirement because the issues are
      the same as those already dealt with in allowing multiple PHP threads to
      process requests from different sessions at the same time.
      
      I have sought to make the progress meter nice and generic, so it can be used in
      the other bulk imports and exports too.
      
      Paradoxically, these changes don't just make the import seem to be faster, it
      actually is.. at least in the case of users and groups.
      
      Times for importing 1000 users, groups and memberships, averaged over 3 runs
      each (Wall time, not CPU time - but the relationship is the same).
      
                      Without Progress     With Progress
      Users                166s               155s
      Groups                85s                78s
      Memberships           20s                19s
      
      Change-Id: Iec15c57db32c77994edb80c71d65591de51a95e4
      Signed-off-by: default avatarNigel Cunningham <nigelc@catalyst-au.net>
      55a8deb8
  3. 24 Sep, 2014 1 commit
    • Nigel Cunningham's avatar
      (Bug1262928) Add support for filtering upload file types. · b0499974
      Nigel Cunningham authored
      Add support for having a CSV (or other file type) filter
      in the File control generated by Pieforms, and use it for
      the forms for uploading users, groups and group memberships.
      
      Note that support for the accept attribute in browsers is
      patchy. This will have no effect in the Chromium and Firefox
      versions current at the time of preparation, but has been
      successfully tested in Chrome.
      
      Change-Id: Ib5cf42be4e054f31086e9db9cda15473e86df7a3
      Signed-off-by: default avatarNigel Cunningham <nigelc@catalyst-au.net>
      b0499974
  4. 14 Oct, 2013 1 commit
  5. 22 Aug, 2013 1 commit
  6. 04 Jul, 2011 1 commit