1. 15 Jun, 2015 7 commits
  2. 29 May, 2015 1 commit
  3. 23 Apr, 2015 1 commit
    • Nigel Cunningham's avatar
      (Bug785472) Remove unneeded raise_memory_limit calls · 039564a5
      Nigel Cunningham authored
      
      
      On the basis of my profiling and testing over the last week,
      all of the scripts affected by this patch don't need their
      raise_memory_limit calls. Ensure_sanity currently sets the
      memory limit to 128MB, which has been seen to be more than
      sufficient for each of these use cases.
      
      Tests have involved at least 1000 records being imported in
      each case, and sometimes as many as 20,000. I would have
      liked to have tested with some really full profiles (eg
      lots of pictures and content), but am satisfied that there's
      enough margin to cover those cases. In any case, such files
      are often handled using external apps and therefore won't
      be counted toward PHP's memory use anyway. Finally, the
      limit can easily be increased by the user if necessary for
      a particular case.
      
      Change-Id: Ifecc83fd47da51268bae6cbd6960735eb91f9403
      Signed-off-by: default avatarNigel Cunningham <nigelc@catalyst-au.net>
      039564a5
  4. 21 Apr, 2015 1 commit
  5. 14 Apr, 2015 1 commit
  6. 13 Apr, 2015 1 commit
  7. 26 Mar, 2015 1 commit
  8. 17 Mar, 2015 3 commits
  9. 13 Mar, 2015 1 commit
  10. 12 Mar, 2015 2 commits
  11. 03 Mar, 2015 1 commit
    • Nigel Cunningham's avatar
      (Bug1352028) Add a JSON progress bar for bulk operations. · 55a8deb8
      Nigel Cunningham authored
      
      
      This patch adds a JSON progress meter (I'll call it that to avoid confusion
      with progress bars) to the bulk uploading of users, groups and group
      memberships and the bulk export and import of users (LEAP), so the user can see
      the progress of the operation and not just the submit button changed to
      'Processing..' and whatever indication their browser gives while waiting for
      content.
      
      The bulk export and import are minor rewrites, replacing the old iframe based
      progress bar and the associated multiple pages and additional template file in
      the case of the bulk export, and the recursive redirect-to-self of the bulk
      import.
      
      To accomplish the display of the progress bar during the operation, we make the
      PHP session be closed (read only) except when changes need to be made. This is
      for the most part a straightforward change in session.php as it's the only
      direct accessor. In other places, we replace direct accessing of the session
      variable ($_SESSION) with use of the session class ($SESSION) so that it can
      reopen the session, make the change and close the session again.
      
      There is one more aspect to all of this: with previous behaviour, multiple
      requests for the same session would queue, taking the session lock in turn.
      After this patch is applied, they can proceed in parallel, allowing greater
      throughput. There is no additional locking requirement because the issues are
      the same as those already dealt with in allowing multiple PHP threads to
      process requests from different sessions at the same time.
      
      I have sought to make the progress meter nice and generic, so it can be used in
      the other bulk imports and exports too.
      
      Paradoxically, these changes don't just make the import seem to be faster, it
      actually is.. at least in the case of users and groups.
      
      Times for importing 1000 users, groups and memberships, averaged over 3 runs
      each (Wall time, not CPU time - but the relationship is the same).
      
                      Without Progress     With Progress
      Users                166s               155s
      Groups                85s                78s
      Memberships           20s                19s
      
      Change-Id: Iec15c57db32c77994edb80c71d65591de51a95e4
      Signed-off-by: default avatarNigel Cunningham <nigelc@catalyst-au.net>
      55a8deb8
  12. 15 Feb, 2015 1 commit
  13. 09 Feb, 2015 1 commit
  14. 05 Feb, 2015 1 commit
  15. 29 Dec, 2014 1 commit
  16. 04 Dec, 2014 2 commits
  17. 20 Nov, 2014 1 commit
  18. 16 Nov, 2014 1 commit
  19. 13 Oct, 2014 1 commit
  20. 12 Oct, 2014 1 commit
  21. 24 Sep, 2014 2 commits
  22. 22 Sep, 2014 1 commit
  23. 18 Sep, 2014 1 commit
    • Robert Lyon's avatar
      The archiving of submitted pages/collections from groups (Bug #1335670) · 5c57b565
      Robert Lyon authored
      
      
      This patch contains:
      - The export queue system where pages/collections on release from
      submission are added to the export queue table ready to be archived.
      - The export queue admin page showing what is in the queue to be
      exported. The cron runs every 6 minutes. Queue items failed to export
      are also shown here.
      - The archive list admin page, where one can download the generated
      leap2a files for the archived submissions.
      
      In this patch you should be able to add things to the export queue by
      either releasing a sumbission on a group that has 'archive
      submissions' option ticked. This will add the archive to that archived
      submission page, or you can also run a leap2a export from portfolio
      export which will add the export queue and send you an email once the
      export is done.
      
      Things to note:
      - The is a server busy function that stops the export queue from
      running but I'm not too sure if the threshold is too low/high
      - The export queue tries to export the first 100 items each run but if
      resources are fine in handling that easily then the number could be
      higher but I'm not sure of what will be a good number.
      - Currently there is alsoe infrastructure like table columns for dealing
      with releasing submissions from external systems (eg moodle) but that
      functuionality is yet to be built.
      - The checking of server busy in MS windows untested - may need to
      just let MS ignore server busy check as there doesn't seem to be
      standard way to check this.
      
      Change-Id: If4c1d272e9c5d46fbf16b2ff73ceb2687c06ffd4
      Signed-off-by: Robert Lyon's avatarRobert Lyon <robertl@catalyst.net.nz>
      5c57b565
  24. 04 Sep, 2014 1 commit
  25. 03 Sep, 2014 1 commit
  26. 22 Aug, 2014 1 commit
    • Robert Lyon's avatar
      Removed maildisabled and authinstnace options from list (Bug #1352027) · 6b4e9f6a
      Robert Lyon authored
      
      
      The way that bulk user csv upload works the authinstance is set for
      all users on the csv via institution field in the upload form so we
      don't need that column in the csv file.
      
      Same with mail disabled - there is a checkbox on the form that we can
      tick/untick to deal with that.
      
      So I've got the system to not display those options on the choices list.
      
      Change-Id: Ib2b3e2f19ee0f8aeed03c7dc04af076721ce8b41
      Signed-off-by: Robert Lyon's avatarRobert Lyon <robertl@catalyst.net.nz>
      6b4e9f6a
  27. 21 Aug, 2014 1 commit
  28. 18 Aug, 2014 1 commit
  29. 17 Jul, 2014 1 commit