Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev/core#942 fix failure to render names for some activities #14223

Merged
merged 1 commit into from
May 11, 2019

Conversation

eileenmcnaughton
Copy link
Contributor

@eileenmcnaughton eileenmcnaughton commented May 10, 2019

Overview

Set limit for activity_contact retrieval to 0, allowing to retrieve more than 25 activity contacts when rendering the first 25 activities on the activity contact tab

Before

before

After

after

Technical Details

This moves the logic for retrieving the target contacts back into the getActivities function. We are stil not wanting to bypass the ACLs so still using the
api but strictly limiting the number of contacts we retrieve (at the cost of extra queries, but cheap ones).

Some tests added on the Bulk Mail activity.

Comments

@pfigel I think some tests will fail now because I tweaked the test data to include a bulk mail type activity. However, let's see how it performs. I think we could reduce the cheap queries slightly but not doing a get-per-row on non-bulk but if there were an activity with many results that was of a different type it would still cause a slow down

@civibot
Copy link

civibot bot commented May 10, 2019

(Standard links)

Copy link
Contributor

@pfigel pfigel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this locally and got an exception for contacts with activities without a target (see other comment for the likely reason).

Other than that, the target names don't seem to get rendered on the activity tab (both for single and bulk activities). Added/Assigned still works.

// the moment as the bulk activity stuff needs unravelling & test coverage.
foreach ($result as $id => $activity) {
// Note that for the non-bulk ones we might get them in one fetch since we 'kinda know' it won't be an insane number
if (!isset($activities[$id]['target_contact_count']) || !empty($activities[$id]['target_contact_count'])) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be && - I'm getting "Expected one ActivityContact but found 0" for contacts with activities that have no target.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pfigel you are right - the outer function needs the target_contact_count set even when the inner one doesn't - There is a weird (read lack of) separation of concerns here.

I've switched to getting the counts for each activity with one simple group-by query, then using a with-acl api call to get the first name for each row with one or more targets.

This could lead to a mismatch - ie they see there are '4 more contacts' but they can't 'see' who they are. I think this is the right behaviour actually

@pfigel
Copy link
Contributor

pfigel commented May 10, 2019

Tested with the latest changes, results:

  • Performance of /contactactivity is now better than under 5.7 (700ms vs. 1.4s) 👏
  • Added By/Target/Assigned to now all display correctly
  • I think there are some inconsistencies or possibly unintended changes with usage of display_name vs. sort_name. Most of that isn't due to this PR, so I suppose it could be handled separately:
    • 5.7: sort_name is used for all three fields (added/target/assigned)
    • master: sort_name is used for target, display_name for added/assigned
    • this PR: display_name is used for all fields (which is consistent, but not sure if an intended change?)
  • (This probably explains some of the test failures as well:) For contacts with no activities, invalid SQL is generated:
Backtrace
May 10 15:09:11  [info] $Fatal Error Details = Array
(
    [callback] => Array
        (
            [0] => CRM_Core_Error
            [1] => handle
        )

    [code] => -2
    [message] => DB Error: syntax error
    [mode] => 16
    [debug_info] =>
      SELECT activity_id, count(*) as target_contact_count
      FROM civicrm_activity_contact
      INNER JOIN civicrm_contact c ON contact_id = c.id AND c.is_deleted = 0
      WHERE activity_id IN ()
      AND record_type_id = 3
      GROUP BY activity_id [nativecode=1064 ** You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')
      AND record_type_id = 3
      GROUP BY activity_id' at line 4]
    [type] => DB_Error
    [user_info] =>
      SELECT activity_id, count(*) as target_contact_count
      FROM civicrm_activity_contact
      INNER JOIN civicrm_contact c ON contact_id = c.id AND c.is_deleted = 0
      WHERE activity_id IN ()
      AND record_type_id = 3
      GROUP BY activity_id [nativecode=1064 ** You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')
      AND record_type_id = 3
      GROUP BY activity_id' at line 4]
    [to_string] => [db_error: message="DB Error: syntax error" code=-2 mode=callback callback=CRM_Core_Error::handle prefix="" info="
      SELECT activity_id, count(*) as target_contact_count
      FROM civicrm_activity_contact
      INNER JOIN civicrm_contact c ON contact_id = c.id AND c.is_deleted = 0
      WHERE activity_id IN ()
      AND record_type_id = 3
      GROUP BY activity_id [nativecode=1064 ** You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')
      AND record_type_id = 3
      GROUP BY activity_id' at line 4]"]
)
May 10 15:09:11  [info] $backTrace = #0 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Core/Error.php(236): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(922): CRM_Core_Error::handle(Object(DB_Error))
#2 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB.php(985): PEAR_Error->__construct("DB Error: syntax error", -2, 16, (Array:2), "\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...")
#3 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-2, 16, (Array:2), "\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...")
#4 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -2, 16, (Array:2), "\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...", "DB_Error", TRUE)
#5 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB/common.php(1907): PEAR->__call("raiseError", (Array:7))
#6 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB/mysqli.php(933): DB_common->raiseError(-2, NULL, NULL, "\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...", "1064 ** You have an error in your SQL syntax; check the manual that correspon...")
#7 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB/mysqli.php(403): DB_mysqli->mysqliRaiseError()
#8 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB/common.php(1216): DB_mysqli->simpleQuery("\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...")
#9 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB/DataObject.php(2415): DB_common->query("\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...")
#10 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/packages/DB/DataObject.php(1607): DB_DataObject->_query("\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...")
#11 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Core/DAO.php(439): DB_DataObject->query("\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...")
#12 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Core/DAO.php(1414): CRM_Core_DAO->query("\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...", TRUE)
#13 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(753): CRM_Core_DAO::executeQuery("\n      SELECT activity_id, count(*) as target_contact_count\n      FROM civi...", (Array:1))
#14 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Activity/BAO/Activity.php(2483): CRM_Activity_BAO_Activity::getActivities((Array:9))
#15 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Activity/Page/AJAX.php(418): CRM_Activity_BAO_Activity::getContactActivitySelector((Array:9))
#16 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Core/Invoke.php(277): CRM_Activity_Page_AJAX::getContactActivity()
#17 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Core/Invoke.php(85): CRM_Core_Invoke::runItem((Array:12))
#18 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/CRM/Core/Invoke.php(52): CRM_Core_Invoke::_invoke((Array:3))
#19 /Users/patrick/buildkit/build/coredev/sites/all/modules/civicrm/drupal/civicrm.module(444): CRM_Core_Invoke::invoke((Array:3))
#20 /Users/patrick/buildkit/build/coredev/includes/menu.inc(527): civicrm_invoke("ajax", "contactactivity")
#21 /Users/patrick/buildkit/build/coredev/index.php(21): menu_execute_active_handler()
#22 {main}

@eileenmcnaughton
Copy link
Contributor Author

Thanks @pfigel I have fixed the bug on no activities that you (and the tests) picked up

I think I'd rather leave the sort_name for a follow up. I want this to get a 5.13 drop sooner rather than later (my own local issues delayed me addressing this quicker) but the other can be a bit slower.

Regarding the speed - it's good that the page you were testing was 50% faster. I would note that is something of a by-product since the main goal was to make it faster in cases the query was slow due to the unindexed joins - that was happening when there were lots of activities for a given contact - which is where we we seeing the tab performing extremely slowly

SELECT activity_id, count(*) as target_contact_count
FROM civicrm_activity_contact
INNER JOIN civicrm_contact c ON contact_id = c.id AND c.is_deleted = 0
WHERE activity_id IN (' . implode(',', array_keys($result)) . ')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just thinking should this really be a param? not sure that you can actually inject anything but meh

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried but commaSeparatedInteger won't pass through in that way

  • however it is definitely safe as they are the array keys from the api array

@seamuslee001
Copy link
Contributor

I have confirmed this fixes the last of Patrick's issues so adding merge on pass

@seamuslee001
Copy link
Contributor

@eileenmcnaughton test fails look related

@eileenmcnaughton
Copy link
Contributor Author

Ok turns out the use of sort_name was tested so no letting it slip - re-fixed

Overview
----------------------------------------
Set limit for activity_contact retrieval to 0, allowing to retrieve more than 25 activity contacts when rendering the first 25 activities on the activity contact tab

Before
----------------------------------------
![before](https://user-images.githubusercontent.com/336308/57439801-e42a0580-729a-11e9-80a1-45df93d0c5eb.jpg)

After
----------------------------------------
![after](https://user-images.githubusercontent.com/336308/57439960-39fead80-729b-11e9-9701-acd79ff73497.jpg)

Technical Details
----------------------------------------
This moves the logic for retrieving the target contacts back into the getActivities function. We are stil not wanting to bypass the ACLs so still using the
api but strictly limiting the number of contacts we retrieve (at the cost of extra queries, but cheap ones).

Some tests added on the Bulk Mail activity.

Comments
----------------------------------------
@seamuslee001
Copy link
Contributor

Test fail unrelated

@seamuslee001 seamuslee001 merged commit 6ed8da0 into civicrm:5.14 May 11, 2019
@eileenmcnaughton eileenmcnaughton deleted the 5.14 branch May 16, 2019 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants